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PREFACE 


With escalating demands for information, the survey question- 
NONE 48 gaining more and more prominence as a took of research. 
Questionnaine design, many researchers insist, remains in the 
nxeakm of art, eluding reduction to any simple collection of 
scientific principles. Yet, as for any art, there are basic 
SkRikls which can be Learned, and which need only be supplLement- 
ed by common sense and a certain degree of scientific sophisti- 
cation. 


This handbook has been prepared as a reference manual for the 
Series Of workshops and seminars on basic questionnatre design 
which are conducted by the Federal Statistical Activities Sec- 
netariat and the Census and Household Surveys Methodology 
Division of Statistics Canada. These workshops and seminars 
are held several times each year in response to requests som 
Auwivey sponsors An the federal government. Although sntended 
primarily for these presentations, the handbook was also pre- 
pared to be seks-contained where the reader has not attended a 
workshop or seminar. Of course, the material contained within 
4s enhanced by such attendance. 


The handbook is intended for general use by persons ucth Limcted 
experience in survey research, and who are faced uxth the task 

of developing a questionnaire. The scope of the handbook 44 
nestricted to those topics in survey design that relate specisic- 
ally to the development of a questionnarrie. As a result, topics 
such as statistical analysis, interviewer training, and sample 
design are not given in detail, except as they relate to question- 
naine design. The treatment of the various topics 4s non-technical 
but neferences are provided for those who uish to pursue any topcc 


in greater detarct. 
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The following members of the Federal Statistical Activities 
Secretariat contributed to the preparation of this handbook: 
Cynthia P. Baumgarten (editor), Charles S. Lyon (Sections 1,2, 
and 3), Philip R. Stevens (Sections 5 and 9), and Edward J. 
Wilhelm (Sections 7 and &, editor). In addition, Ruth M. Corbin 
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Swain of the Census and Household Surveys Methodology Diviscon 
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MEETING INFORMATION NEEDS 


A common procedure in research is to realize that a problem exists and 
without a particularly detailed analysis of that problem, to begin to 
draft a questionnaire. Often, this is associated with the general idea 
to "do a survey". There are two basic errors in this process. The first 
is in attempting to develop a questionnaire at too early a stage in the 
research. Many questionnaire design problems depend for their solution 
on the thinking and analysis put into the problem even before the consi- 
deration of a questionnaire and certainly before the drafting of a pre- 
liminary version of a questionnaire. The second basic error lies in 
deciding immediately that a survey is the means for solution of the pro- 
blem. A danger exists in making this decision early in the planning in 
that the problem itself may get lost within the various stages of survey 
development, that the survey may not even solve the whole problem, or that 
the survey may be aimed at solving the wrong problem. 


The research method as proposed below is presented as a problem solving 

process, requiring that knowledge and analysis be applied long before a 

questionnaire is considered or a procedure for data collected is decided 
upon. 


1.1 Recognition that a Problem Exists 


The first and most basic step in problem solving is the realization that a 
problem may exist. The problem may be an unexpected deviation from what 
should be occuring in a process or the necessity to evaluate a government 
socio-economic program or a. specific. request for data. . An initial ‘state- 
ment of the problem may be broad or narrow, general or specific. 


1.2 Clarification and Analysis of the Problem 


The recognition and initial statement of a problem does not necessarily 
mean that one can begin immediately to solve it. Usually, the problem 
is not yet stated in researchable terms. For example, "evaluation" may 
have many different meanings, some of which may be researchable and some 
of which may not be. Broad terms such as "the transportation problem", 
"attitudes concerning capital punishment", and "service to the public" 
are often all that the research has as a starting point. 


The assimilation of background information concerning the problem then 
becomes essential as a means of clarifying and expanding it into its 
various components. The formulation of the research problem involves 
consideration of some of the following details: a statement of what the 
problem is, a statement of what the problem is not (these aid in the defi- 
nition of the limits of the problem), a necessary step in the determina- 
tion of whether the problem can be solved (either fully or partially and 
in what depth), possible causes of the problem, the relationship of the 
problem to previous research, the relationship of the problem to 


pre-determined goals, the relationship of the problem to a theoretical 
framework, and alternate hypotheses. The researcher is now in a position 
to sort out the many aspects of the problem, to choose among them and to 
specify them in terms which can be research (operational terms). 


The foregoing may require background reading, discussion With Experts. on 
intense introspective study. However, the more detailed the researcher 
can be at this point, the better defined will be the problem. As a 
result, the determination of the steps which follow will be more simpli- 
fied and directed, leading eventually to the resolution of the problem or 
the decision that it can't be resolved. 


1.3 Determination of Information Needed 


Once the problem has been clarified and analysed, the researcher must 

then determine whether further information is required to answer the pro- 
blem. If so, the questions such as what information is needed, from whom 
or where the information should be obtained, the uses to which the infor- 
mation will be put and how the information will contribute to solving the 
problem can be answered. Answering these questions will require that the 
objectives for data collection be stated and linked directly to the broader 
research problem. Since some objectives will be more important than others, 
the researcher should assess the priority of each objective. 


The above is usually an iterative process. Once consideration of the in- 
formation needed takes place, it may be found that the original problem 
was not specific enough to fully indicate what information would answer 
tC. “lites iterative property is shown in figure’ 1.1 


1.4 Means of Acquiring Information 


In general, data may be obtained from three sources: documents, records 
or other existing information: by observation and by questioning. 


Documents, records and other existing information can take many forms in- 
cluding letters, family records, institutional records (hospital records, 
for example), administrative records (sales figures, bank accounts, per- 
sonnel records), published and unpublished reports, and data collected 

for previous studies. All organizations, for example, maintain some re- 
cords on their personnel and products and these records may provide data 
for study. (The records are distinguished from data as they are produced 
for reasons other than aggregate study.) Similarly, data which have been 
gathered in previous studies, surveys and so on may be amenable to second- 
ary analysis. The Census of Population and Housing is a primary example. 


Documents and other existing information may often replace the need for 
collecting new information, and at times may provide the only source of 
information, for example, where respondents are no longer alive, or where 
the information could not possibly be recalled in sufficient detail. 


There are, of course, certain characteristics of documents, records, and 


existing data which must be taken into account to ensure the information 
is Suitable for the purpose. Timeliness, scope of coverage of informa- 
tion and population, reliability of recording, and the definitions and 
meanings of terms must be known and evaluated in retrospect, in much 

the same way as plans for a new collection are analyzed. 


A concrete example of the use of documents is a study which was carried 
out in a major metropolitan area. The objective was to determine hospi- 
tal service areas and the manner in which people in the city used the 
various hospitals. The first decision was to conduct a sample survey 

and ask residents directly about their use of the hospitals. A pretest 
Showed that only a small proportion of the population used the hospitals, 
that they had difficulty remembering dates of use, names of hospitals and 
reasons. In fact, people who had been admitted to hospitals had only 
vague technical knowledge of why they had been admitted and what services 
were used. Faced with insurmountable problems of sampling, knowledge, 
recall and accuracy, it was decided to use hospital records. This had the 
advantage of completeness, accuracy and records which were available for a 
long period of time. It was possible to obtain acceptable results from a 
small number of records (2,500 or less per hospital) which showed the ser- 
vice areas, reasons for use and characteristics of the user population. 


The cost of the study was reduced from $200,000. to approximately $25,000. 
Not all uses of documents are so dramatic but all possibilities should be 
considered. 


The method of obtaining data by observation also includes a variety of ap- 
proaches. It generally covers methods which do not involve direct ques- 
tioning of the subjects of observation, but the concept may include in- 
direct methods of questioning, particularly where the observer is partici- 
pating in some kind of social group in order to understand the workings of 
the group. In general, however, observation is considered to be a form of 
data-gathering where the subject of observation is not directly questioned 
to obtain the information. 


In some very obvious instances, observation must be used. For example, no 
questioning is possible in natural and physical sciences inquiries. Where 
the respondent is not able to speak (small children), observation may be 
the only viable method. In other instances, observation may provide ade- 
quate data or may be desirable over questioning. For example, traffic 
counters provide a more accurate measure of flows and volumes than would 
direct questioning of respondents about their movements. Similarly, 
videotape or still photographs will provide more accurate information 
about congestion than will questioning. Behaviour may differ with the 
presence of an interviewer and often, questions about behaviour are ans- 
wered with socially desirable responses (the answer conforms with what is 
expected as "proper" social behaviour, e.g., one does not readily admit to 
abusing children or alcohol, so the incidence is underreported). Observa- 
tion often provides quite different results. Observation may be obtrusive 
or unobtrusive in the sense that subjects are aware that they are being 
observed, or are unaware. In fact, some observational techniques gather 
data from traces or remains of physical evidence in much the same way as 


do archeologists. A questionnaire study of alcohol consumption in a 
legally "dry" area came to the conclusion that people in the area drank 
little or no alcohol. An observation study of garbage cans indicated a 
significantly higher consumption. 


The major difficulties with observational techniques are the extent to 
which observers affect the behaviour of those being observed by their 
presence, the development of an analytic scheme to adduce meaning to the 
observations, and the development of accurate recording sheets to record 
the behaviour. There are various methods to counteract the obtrusiveness 
of observation, and meanings and recording schemes are usually based on 
theoretically or empirically grounded hypotheses. The most simple are, 
of course, those which require only counting to provide a cross-sectional 
view of an existing situation. As observation is often more accurate and 
less costly than questioning, observational methods may be considered as 
possible alternatives, or for use in conjunction with other techniques. 


Questioning procedures cover a range of information gathering which re- 
quires participation of the respondent in providing the information 
directly. For our purposes, questioning includes both interview and 
self-administered response forms (commonly called questionnaires). Ques- 
tioning may take a form ranging from unstructured exploration of topics 
such as obtaining life histories or exploring psychologies through psycho- 
analysis, through to highly structured formats such as a self-administered 
mailed questionnaire which requires the respondent to choose one of a 
fixed set of answers. 


The amount of structure depends to a large extent on the nature and 
extent of the researcher's knowledge and the range of responses which 
will adequately answer the questions. Where, for example, little is 
known about a topic, the responses are likely to be more open to the 
interpretation and choice of the respondent. At an extreme, respondents 
are given total freedom to develop answers, as in psycho-analysis. The 
main emphasis of the following chapters is on questioning through the use 
of structured interviews of self-administered questionnaires and further 
discussion of these approaches will be left for those chapters. Often, 
however, there are situations where insufficient knowledge of a topic 
requires an unstructured approach to develop more information. In addi- 
tion, the structured interview or questionnaire is most amenable to use 
in a "mass interview" situation, where the goal is mainly to develop 
generalizations about an aggregate rather than to understand a Specitie 
Situation or develop individual case information. Obviously, generaliza- 
tions cannot be made based on non-standardized data, but there are ins- 
tances where generalizations are not the goal and an unstructured format 
may be considered. Typically, these situations arise in policy formula- 
tion where experts or holders of power may exert influence regardless of 
mass opinion, or where their guidance or expertise is relevant. It is 
hardly relevant to know, for example, what the majority of students want, 
when educational policy makers in power have a different view. 


One type of unstructured questioning format deserves mention here because 
it is used widely. Most forms of questioning imply a one-to-one inter- 
action between researcher and respondent. The "group interview", on the 
other hand, treats a number of people at once, usually raising a topic 
and asking the group members to express their opinions. This format may 
be used as a preliminary attempt to understand the thoughts and opinions 
of people, and is more economical than extensive exploration of the same 
topic with the same number of respondents individually. The existence of 
the group poses some unique problems in interpretation of the results, in 
that social influences on respondents are quite different from a one-to- 
one encounter, and it is impossible to judge such things as knowledge, 
salience and intensity of feeling of individual respondents. The tech- 
nique is, however, a relatively economical way to formulate hypotheses 
about a topic, and some researchers believe that the effect of group dy- 


namics brings out information which may not be obtained in individual 
interviews. 


Not all of the available methods of gathering information have to be con- 
sidered for every study. Preliminary evaluation of the problem serves to 
eliminate many from consideration at the outset. There are usually 
several which remain as possibilities, each with their own characteris- 
tics, and a choice must then be made. While this booklet deals with a 
necessarily limited scope, the researcher should be aware of alterna- 
tives, and reject them on grounds of choice rather than ignorance. 


The choice of a particular technique for asking questions and a precedure 
for data collection will depend on the type of information desired, the 
persons from whom or the location from which it is collected, the purpose 
of the collection, the specific uses and analysis, and any resource con- 
straints (cost, time, or availability of manpower). In fact, the con- 
straints may be such that the researcher is forced to choose a less than 
ideal technique or procedure, to choose to satisfy only those objectives 
with» relatively hich priority: orkto revise whe objectivesnot, tic 
research. 


In the case where the research problem is the evaluation of a government 
program, the determination of the method of data collection may be sim- 
plified and more efficient if the evaluation of the program is considered 
at the time of the program planning itself rather than as an afterthought. 


1.5 Collection of Data and Their Assembly into Useable Information 


Once data have been collected and processed, the researcher is in a posi- 
tion to analyse the data relative to the original research problem. The 

analytical process will be greatly simplified and the information collec- 
ted more relevant to the problem, if one has considered the uses to which 
the data will be put prior to the data collection itself. 


A final step is the interpretation of the results. Conclusions are drawn 
and recommendations made with direct relation to the resolution of the 
research problem and the original hypotheses. This in turn may suggest 


further research. 


Figure 1.1 below outlines the research method described above as a 
ciccular process. 


Figure 1.1: The Research Method 
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THE SURVEY QUESTIONNAIRE 


The main topic of this booklet is questionnaire design of a particular 
type. Obviously, a questionnaire (in the sense of a set of questions or 
some form of recording instrument) is necessary whether the source of 
data is documentary, observational or a result of questioning. Within 
the range of techniques used in questioning, the "questionnaire may range 
from an undefined set of topics through to a highly structured set of 
questions with no options for response other than those listed. The 
Survey questionnaires on which we focus falls at the structured end of 
the range. It is a specific instrument which is structured so that the 
Same, Standardized questions are asked of everyone relevant to the study. 
An attempt is made to make interviewing a routine task which does not 
require interviewers to make decisions and interpretations in the course 
of the interview, and in the most standardized or "mechanized" form, the 
questionnaire is "self-administered", i.e., filled in completely by the 
respondent without the aid of an interviewer. 


The survey questionnaire has an advantage over some other techniques in 
that it provides standardized data which are amenable to aggregate anal- 
ysis to describe group characteristics. It may be administered to large 
numbers of people without the need for highly creative interviewers 
because the interviewer is not required to participate in the development 
of the information. Usually, the cost per interview is less the more 
structured the questionnaire becomes. 


On the other hand, the survey questionnaire is by no means a universal 
tool for gathering information. There are times when data are not avail- 
able to respondents (e.g., certain types of administrative data cannot be 
obtained from respondents) or when data may be expected to be distorted 
(e.g., reports on behaviour) particularly where illegal or socially unde- 
sirable behaviour is involved. There is also lack of agreement on whe- 
ther attitudinal information is amenable to collection through survey 
questionnaires though certainly there are techniques which are used to 
gather opinion and attitude information through surveys. There are res- 
trictions in the type of data which may be obtained, and the exploration 
of topics from the point of view of the respondent is severely limited. 
Finally, although the cost per interview may be relatively low, mass 
interviews usually cover a large population, and the structured nature of 
the questionnaire requires a great deal of thought and planning as well 
as strict field controls to ensure that questions are appropriate and the 
data are accurate. The decision to use a survey questionnaire to obtain 
data should, therefore, not be taken lightly. 


2.1 Use of the Survey Questionnaire 


The survey questionnaire is generally well suited to situations where the 
topic can be restricted in scope, and requires “factual” om “objective” 
information needed for the purpose at hand, should ask only questions 
which the respondent is known to be able to answer, require only a ‘yes' 
or 'no', a number, or similarly simple response, and be amenable to ans- 
wer with a truthful and unbiased response in neutral terms. Much of 
questionnaire design is devoted to attempting to achieve such objective 


results. 


PLANNING A SURVEY 


The design of a survey questionnaire is intimately related to and flows 
from the overall survey design. The following section outlines the plan- 
ning stages which lead to the design and testing of specific questionnaire 
items (questions). 


S, LU wbrepsaratvion of Objectives 


The preparation of objectives for the survey is perhaps the most impor- 
tant step in designing a survey questionnaire. It is also the point on 
which most designs fall short. There are two general cautions to be made 
about preparing survey objectives: survey objectives are not necessarily 
as broad as overall information needs; survey objectives should never be 
stated in terms of "gathering information". 


The objectives of the survey evolve from consideration of the general 
problem, but usually the general problem covers a wider range of topics 
and considerations than will the survey. The questionnaire is naturally 
limited in the extent and type of data which can be collected, so that 
many aspects of a topic cannot be considered (though other data and tech- 
niques may be used in conjunction with a survey to cover other aspects) 
and several different surveys may form parts of the same project. 


It is difficult to deal with the setting of objectives in abstract form, 
as each problem will require unique consideration. In general, however, 
the range of possible objectives should be written down and considered 
for suitability.) ‘Tt has. been found that anseffective way .to statesapiee- 
tives is in terms of the nature of conclusions: one ‘wants.to makes’ “For 
example, a vague objective such as "to gather site utilization data for 
site planning", can be clarified by considering what the planning deci- 
sions will be: These may be: to decide whether to expand the parking 
list; to decide whether to clarify the interpretation facilities so that 
more people understand them; to determine whether people understand the 
interpretation; to advertise the site in potential tourist areas ;sor any 
number or combination of other objectives. Consideration should also be 
given to the accuracy of the data required to meet the objectives (or to 
make the decisions). For example, if one of the objectives is "to decide 
whether to expand the parking lot", an auxiliary question the research 
will have to ask is "under what conditions will the decision to expand 
the parking lot be 'yes'?" The answer may be something like, "if the 
overflow is 50 vehicles or more at any time", or "if there is any over- 
flow", or, "if the overflow reaches 10 or more twice during any day". 

Any number of other criteria may be in the mind of the researcher or 
planner and those should be made as explicit as possible at the beginning. 
Where decisions are at issue, the decision criteria will become more 


important and more specific as reiteration through the various design 
stages takes place. 


Finally, it is usually profitable to consider the importance of the accu- 
racy of estimates as part of the setting of objectives. Estimates are 
usually a feature of survey data because some form of sampling is almost 
always involved and estimates always involve a range of error within 


which one is uncertain whether the conclusion is correct. That Ae at 


the parking lot example, the measurement of the decision criterion (or 
Criteria) may be subject to error due to sampling procedures and it is, 
therefore, important to consider what the extent of tolerable error is. 
This may affect the cost, collection method, sample size and sample de- 
Sign in a variety of ways. In operational terms, tolerable error can be 
evaluated by asking oneself (or the planner) "what are the consequences 
of making an incorrect conclusion?" This can be either to expand the 
parking lot when it is not necessary (because the estimate is higher than 
the actual criterion) or not expanding when it should be (i.e., the esti- 
mate is lower than the actual criterion). In the example above, the 
tolerable range of error, or the "chance" one is willing to take of mak- 
ing an incorrrect conclusion from the data, may be quite large. In other 
circumstances, for example where a survey may be done to determine the 
linkage between exposure to carcinogens and the incidence of cancer or 
other morbidity, an incorrect conclusion may have extremely serious con- 
sequences. While it may not be necessary in every project, the services 
of a competent sampling statistician are often invaluable at the objec- 
tive setting stage and should be definitely considered for all large or 
complex projects. 


It should be noted that these objectives may not be mutually compatible, 
so that designing a single survey to cover all of them at once may be im- 
possible. While some of the objectives may be jointly met with the same 
data, others may require quite different approaches, and decisions may 
have to be made regarding which topics are most relevant, which are most 
important, and perhaps which may be acted upon. Budget and time con- 
straints enter into this decision, as does consideration of the relative 
usefulness of information in practical terms. Obviously, the-béest infor 
mation in the world about problems over which one has no control has 
questionable value. 


Thus, the relevance of each topic or potential objective must be deter- 
mined and evaluated. The determination of objectives is often an itera- 
tive process, in which some objectives are stated, evaluated and chosen, 
examined for feasibility, operationalized in a draft questionnaire, then 
restated and re-evaluated until a final set is determined. No matter how 
many circuits are taken and how many different objectives are considered, 
at each stage the statment of survey objectives provides the core defini- 
tion to everyone concerned, and provides a reference point to ensure that 
the more detailed development of plans is relevant. 


3.2 Development of General Plan for the Survey 


There are a number of factors which are involved in the general survey 
plan, and these flow naturally from the objectives. The general survey 
plan provides the outline of decisions to be followed when the survey is 


detailed and conducted. 


3.2.1 Subject Content 


The general subject content evolves from the statements of use of the 
data. Usually one begins with a broad statement of subject areas. For 
example, if one of the objectives is to decide where to place advertise- 
ments for an attraction, one of the subject areas is likely to be the 


place of origin of visitors. At this stage, specific question wording 

is not at issue, as this will be affected by other considerations includ- 
ing the survey population, collection method and level of detail required 
to specify or "measure" place of origin. In other words, there is usual- 
ly not a straightforward transfer of objectives into questions for a 
questionnaire, but a logical development of dit ferent aspects of the plan 
so that questions are translated from general statements into operational 
descriptions of the data required. 


3.2.2 Survey Population 


The survey population is the general group or category of respondents who 
are relevant to the survey objectives. The population may consist of in- 
dividuals, households, vehicles, parties, businesses and so on, which have 
certain characteristics. A specific consideration of the survey popula- 
tion must be made, and the population defined specifically. Decisions 
about which population is appropriate to meet the desired objectives will 
have to be taken. To follow the advertising example, decisions will have 
to be made about what, specifically, is the nature of the population that 
can provide information on where to advertise. This may be site visitors, 
residents of locations where site visitors originate, residents of loca- 
tions where site visitors do not originate, all tourists or some other 
category. 


Jsvaxo pLze oF the Survey 


This is determined by the accuracy and confidence level required, the num- 
ber and size of sub-populations for which information is required, the 
sample design, the complexity of information and, ultimately, possible 
budget and time limitations. Here as in other aspects of design, one can 
expect an iteration between the setting of objectives, consideration of 
accuracy, restating and further specifying objectives, and so on, until 
the objective is clear enough to decide on the size required. In the pro- 
cess, budget and time considerations serve as limiting factors. For exam- 
ple, a particular combination of accuracy and topic coverage requirements 


may be too expensive. Objectives may have to be modified or dropped, or 
subjects may have to be redefined. 


3.2.4 Collection Methods 


Answers to questionnaires are generally obtained either by an interviewer 
asking questions and recording responses (interview), or by respondents 
reading the questions and recording their own responses. (A variation on 
the latter type is the "diary", where respondents record events as they 


happen. The diary is, however, structured to record only events of inter- 
est to the study.) 


An interview may be conducted over the telephone or face-to-face. Face- 
to-face interviews may be conducted in many locations. Self-administered 
questionnaires or diaries may be mailed to addresses, hand delivered, 


handed out at different sites or simply left in certain locations. The 
responses may be picked up, mailed back, dropped in a return slot, tele- 
phoned to a certain location, or related to an interviewer. Combinations 
of any of these can be used in a single survey. 


Five factors are involved in the choice of collection methods: the Pree 
bable response rate, the expense, the accuracy required, the complexity 
of information it is necessary to obtain, and the speed with which 
answers must be obtained. There are no hard and fast rules to follow, 
but there are some general guidelines. Usually, higher response rates 
and more complex information canbe obtained with interviews than with 
self-administered questionnaires. Telephone interviews fall somewhat in 
between face-to-face and impersonal methods. On the other hand, costs 
are higher per questionnaire as personal contact increases. Obviously 
the complexity of information required and the number of completed ques- 
tionnaires required, plus considerations of bias due to non-response will 
be major factors in determining whether a given method is likely to 
achieve results adequate to meet the objectives. One, or a number of 
collection methods may be considered and the trade-offs between costs and 
objectives evaluated. 


3.2.5 lhe Deatt Plan 


A draft plan should be drawn up which states the specific survey objec- 
tives, decisions on the foregoing factors, alternatives and considera- 
tions which have to be made to take a decision, and estimates of timing 
and costs involved in following the plan. The draft plan is necessary 
even for small surveys, though perhaps not as many alternatives will 
have to be considered. 


In some cases, it may be possible to decide on all of the relevant fac- 
tors at the ovtset. “‘In“others, however’ at ‘canbe expected that the plan 
may undergo several drafts as alternatives are checked for feasibility 
and topic areas more firmly defined. The draft plan is the basis on 
which the survey will be modified and designed in detail. 


3.3 Exploration and Testing of Sources and Methods 


To some extent, all features of the draft plan are tentative until they 
are explored for feasibility. Exploration of the situation envisaged in 
the draft plan may uncover different sources of information which are 
more fruitful than the ones initially chosen. The supposed population 
may prove not to have information on the general topic areas of interest. 
The methods of sampling, data collection and so on may not be feasible 
(there may be no frame - a list of the survey population - to sample 
from, there may be no telephones to call to, the respondents may not 
speak the same language) in the actual situation. The exploration and 
testing phase is therefore important to ensure that major elements of the 
plan will work before a great deal of effort is put into final design 
details. Note that even at this stage, the specific questionnaire de- 
sign need not have (and some would say should not have) begun. 


a. “Revision of Plan 


This stage takes into account and modifies the parts of the plan which are 
inadequate or unsatisfactory. Revisions may have major ramifications 
extending back to, and requiring modifications to be made to, the state- 
ment of objectives as well as to parts of the survey plan. Inadequacy of 
an essential part of the survey plan, discovered only during the full- 
scale survey, may jeopardize the usefulness of the results. 


3.5 Specific Concepts and Operational Definitions 


Once we are satisfied that the draft plan is workable, the topics to be 
covered can be reviewed against the objectives to specify their exact 
meaning. 


Specific concepts and working definitions should be developed and applied 
to all possible types of survey situations to ensure their relevance and 
clarity. For example, even such a simple item as "income" should be de- 
fined as "salary", "total family income", etc., as appropriate. Questions 
can then be developed which will ask for the exact information required, 
e.g.: "total family income before deductions in 1979". As the specific 
questions, working definitions and concepts become more fully developed, 
the objectives, content, hypotheses, etc., often become subject to further 
refinement. This is an iterative (or circular) process, as various drafts 
of a questionnaire are usually required before a final version is 
produced. 


TYPES OF QUESTIONS IN SURVEY RESEARCH 


We have seen that an essential first step of questionnaire design is a 
well-defined statement of what information is required. Nonetheless, the 
researcher is often left with considerable freedom and creative responsi- 
bility in how to ask questions. The tools available to him/her, to aid 

in question wording, include a classification of questions into recogniz- 
able types. Consider, for example, the hypothesis that Canadians today 
consider "National Unity" a rather unimportant political issue. What, 
specifically, does this imply? Each of the following is a possible opera- 


tionalization of this hypothesis. Each is inspired by a different ques- 
tion type: 


When asked to freely generate all the current 
political issues which they consider impor- 
tant, few respondents would mention National 
Unity; 


More respondents would answer "No" rather 
than "Yes" when asked if they consider 
National Unity an important political issue; 


When asked to choose the most important 
political issue facing Canada today from a 
given list, few respondents would choose 
"National Unity"; 


When.-askéd“toscheck. off; .tromiacgivenalast, 
all the political issues which they consider 
important, few respondents would check off 
"National Unity"; 


When asked to rank a given set of political 
issues in order of importance, respondents 
would not rank "National Unity" very highly; 


When asked to rate the importance of "National 
Unity" ona scale *from'l to10, respondents 
would not rate its importance very highly. 


In the sections to follow, the questions types will be identified which 
have shaped the hypothesis of "“unimportance" in the several ways identi- 


fied above. 


A question typology also defines the options for the format and analysis 
wisdaca. "for, once a particular type of question is selected, the sta- 
tistical analyses available for consideration are immediately determined. 
Illustrations to support this point will be presented throughout. 


In summary, then, identification of question types lends structure to 


questionnaire design in two ways: 


it classifies the various methods of verbalizing 
requests for information; and 


= hi gaa 


it classifies the numerous modes of data analysis 
one can call upon. 


The first and perhaps most obvious distinction to be made in the classi- 
fication hierarchy is that of the "open-ended" versus "closed-ended" 
questions, which the following section discusses. 


4.1 Open-Ended vs. Closed-Ended Questions 


The distinction between the open-ended and closed-ended question concerns 
the degree of freedom accorded the respondent in choosing his answer. An 
open-ended question allows him to generate a response on his own, and in 

his own words; a closed-ended question restricts him to choosing among 

- alternatives specified in the questionnaire. 


Here is an example of a question whose content can be adapted to either 
type: 


Example 4.1 
Open-ended: 


What Gypes. of music do you ‘most. Pike to Listen to? 


Closed-ended: 


Here is a list of different types of music. Please indicate which types 
you most Iuke to listen to: 


ey Jazz 

ces) Popular 

7) Folk 

3 Classical 

C- Ey Country and Western 

Ps) Rock 

t-awa Blues 

i <3 Other (specify) 

oo. None. I don't like to listen to any type of music. 


There has been also identified, in the literature on survey methodology 

a question type called the "partially closed-ended" question, used in 
interview Situations. To the respondent the question sounds open. He is 
simply asked, for example, what types of music he most likes to listen 
to. But the interviewer has a pre-coded sheet for his responses. As he 
talks, the interviewer matches the types he mentions against a list. The 
purpose of this question type is mainly the gain in efficiency by having 
responses coded as they are given. To obtain this efficiency, the 


researcher needs to predict the likely range of responses to the question 
- not always a straightforward task. 


Consider for example the possible reactions to the following question 
(adapted from Warwick and Lininger, 1975, p. 135): 


Example 4.2 


How much does your job asan administrator challenge you - in the sense of 
demanding your skills and abilities? 


When approached informally with this question, administrators in an of- 
fice may give a variety of responses, ranging from an adverb description 
of how much they were challenged ("slightly"), to an indication of the 
percentage of skills and abilities exercised ("about 40%"), to a reply 
combining frequency and amount ("sometimes demands a lot, other times 
very little"). Such evident discrepancies in interpretation of the ques- 
tion or in the mode of measurement which the respondent uses to indicate 
"amount", are often described as differences in frame of reference. That 
is, respondents may think along different lines in devising what they 
consider to be appropriate responses. 


The problem in this, relevant to the present discussion, is that the 
questionnaire designer himself may assume a particular frame of reference 
and construct a "partially closed-ended question" accordingly. Thus, he 
may list, for the interviewer's sake, the following options to the 
question above: 


Completely demands my abilities 
Demands most of them 

Demands about half 

Demands some of them 

Demands very few of them 


ONG e ONY fe Ne oN 


But as our informal pre-test illustrated, an interviewer is unlikely to 
find a respondent who spontaneously utters one of these options; indeed 
the sample responses given above demonstrated that a respondent need not 
even think along the same implicit "scale"'. Thus, there is no gain in 
efficiency, and a good deal of potential frustration on the part of the 
interviewer, in having this list of response options to refer to. The 
conclusion one draws is that, without change in wording, this question is 
inappropriate for a partially closed-ended format. 


As indicated in the last paragraph, the partially closed-ended question 
has been defined for the interviewer's sake. What has not been explored, 
but appears to hold potential for further research, is the notion of par- 
tial closure from the respondent's point of view. RaAGaae hers typically 
emphasize how much opportunity for "freedom of expression" an open-ended 
question gives the respondent. But consider the fel oing open-ender: 
"On what days of the week is your garbage collected? Although no list 
of alternatives is read to the respondent, it is reasonable to argue that 
he generates, mentally, a pre-coded checklist i ha hat of the seven 
days of the week. In this way, he “partially closes the question. In- 
deed, one may argue more extremely that, with such obvious response ai~ 
ternatives as days of the week, the question is perceived as being com- 
pletely closed-ended. It then becomes of interest to inquire Seema 
about the respondent's perceptions. When a question is presented to him 


as open-ended, does he perceive complete freedom, some freedom, or no 
freedom at all in the response he is allowed to select? Indeed, under 
what circumstances does he select from a list of mental response alterna- 
tives, as opposed to reporting whatever response first enters his head? 
And how can we systematically tie the psychological difference of those 
two response strategies to objective characteristics of questionnaire 
items? 


So between the two well-defined extremes of open and closed-ended ques- 
tions is an ambiguous middle ground. For the researcher, there is a 
varying degree of confidence about whether partial closure of questions 
would increase efficiency. For the respondent, even open-ended questions 
may lean toward closed-endedness if the response options are obviously 
constrained and small in number (such as days of the week). Thus, the 
respondent has a parallel variation in the extent to which he perceives 
"closure" in open-ended questions. Exploring further this notion of a 
partially closed-ended question - from the respondent's point of view - 
may open new doors in understanding the role of respondent perceptions 
in the survey process. 


For the remainder of the discussions, however, let us set aside this 
ambiguous middle ground, and define a question as either open or closed, 
according to whether or not the alternatives are explicitly presented 

to the respondent. The next section describes a sub-classification 
scheme for closed-ended questions and offers more detail on their analy- 
Sis and use. 


4.2 Closed-ended Questions 


4.2.1 Identifying the Subtypes 


The three simplest variations of closed-response questions are the "two- 
choice", "multiple-choice" and "check-list". These are illustrated 
below, in three sequential questions taken from a 1976 Statistics Canada 
Survey on "Fitness, Physical Recreation and Sport". 


Example 4.3 


Two-choice question: 


30. Is there any sport or physical recreation 
activity in which YOU DON'T PARTICIPATE 
that you would like to start doing 
regularly? 


YES NO GO TO QUESTION 33 


(Print the name of the activity) 


=e 


Multiple-choice question: 


31. In what context would you like to partici- 
pate? (Check one only.) 


Non-competitively 
Competitive non-organized 
Competitive Organized 
Other 


Saez) 1s 


Checklist question: 


32. Why haven't you participated in this sport 
or physical recreation activity? (Check 
one or more). 


EY Its difficult to find others) to 

participate with. 

I am physically unable to participate. 

Not enough time - other leisure activities. 

Not enough time because of work (school). 

There is no opportunity to participate 

near my home. 

QO It costs too much to participate. 

O The available facilities/arena are of poor 
quality, inadequate, not challenging. 

O I don't know how to do it well enough. 

O Other reasons (Specify) 


O 
O 
O 
O 


A further type of closed-ended question is a ranking question, in which 
respondents must arrange all of the given alternatives in an order cor- 
responding to some specified criterion. Here is an example which might 
be used in a study of job search behaviour. 


Example 4.4 


Ranking question: 


Below is a list of some of the ways people go 
about finding jobs. Please rank them in order 
of effectiveness by placing the number (1) beside 
the method you think would be most helpful, a 

(2) beside the method you think would be second 


most helpful, and so on. 


Mailing out resumes ( 
Newspaper or magazine ads ( 
Canada Manpower Centres ( 
Checking with friends ( 
A private placement service ( 
Unemployment Insurance Office ( 
Direct contact with employers ( 


The final type to be considered here is a rating scale, illustrated in 
the following question: 


Example 4.5 
Rating scale question: 
"Please indicate your satisfaction with the bus 


service in the city by circling the appropriate 
number on the scale below." 


1 2 3 4 5 
Very Very 
Dissatified Satisfied 


Though rating scales vary in format, they share the property of repre- 
senting response strength by a graded scale. Now a scale is an abstract 
representation. Thus, the use of one requires an assumption that the 
rating task is "meaningful", the respondents understand the correspon- 
dence between the line on the page and some real-life variable. 


A creative version of a rating scale was introduced by Lawrence Leduc 
(1977), who was obviously attempting to ensure that the rating task would 
be meaningful to respondents. He asked them to indicate their personal 
feeling of warmth towards their province and country by referring to 
temperatures on a thermometer: 


Example 4.6 


"You will see here a drawing of a thermometer. 
It is called a feeling thermometer because it 
helps us measure people's feelings toward 
various things. Here is how it works. If you 
don't have any particular feelings toward the 
things we are asking about, place them at the 
50° mark. If your feelings are very warm toward 
a particular thing, give a score between 50 and 
100 - the warmer your feelings, the higher the 
score. On the other hand, if your feelings are 
relatively cool toward something, place it be- 
tween 0 and 50. The cooler your feelings the 
closer the score will be to zero." 


ap Ss 


a visual rating scale technique involves a card which looks like 
this: 


Example 4.7 


NO 


The respondent is asked to answer in terms of a “biesyes" sa "little veo" 
evelattle nol, .ov.a “big ne" (Paynes (1951s 


4.2.2 Two-Choice and Multiple-Choice Questions 


When a decision among two or more clear-cut alternatives is called ron, 
it is obviously an occasion for a two-choice or multiple-choice question. 
The typical hypothesis under investigation predicts that one alternative 
will be favoured over the rest, or that the percentage of "votes" for the 
various alternatives will match a certain pattern. Frequency counts are 
thus the principal index of data analysis for both two-choice and multi- 
ple-choice questions. The chi-square statistic is available for deter- 
mining the significance of frequency differences. It may also be used to 
test whether the proportion of votes for all alternatives is in line with 
a researcher's prior expectations. 


The validity of the data from two-choice and multiple-choice questions 
rests on a particular assumption - that the alternatives or choices offered 
in the question be mutually exclusive and exhaustive. A respondent must 
be identified with one and only one alternative. Data from the following 
question, for example, could not be meaningfully analysed: 


Example 4.8 


"Tn which of the following categories is your 
age in years?" 


‘ea: 20-30 
ae 30-40 
er 40-50 
oe more than 50 


The categories are neither mutually exclusive nor exhaustive. The list of 
categories could easily be resolved as follows: 


less than 20 
20-29 

30-39 

40-49 

50 or more 


PO fe See 
Nee Nee Nee Nee See’ 


ZO 


Sometimes, for multiple-choice question, collapsed frequency counts are 
employed. A respondent may be asked to indicate whether he is "very . 
satisfied, somewhat satisfied, somewhat dissatisfied or very dissatisfied" 
with his local postal service. Later, the data could be collapsed over 
the first two and last two response categories, to reflect percentages of 
"positive" versus "negative" attitudes toward postal service. 


If the data are going to be collapsed anyway, the reader may ask, why not 
ask a two-choice question in the first place? Shouldn't one use the 
simplest question formatting that will test the hypothesis of interest? 


The simplicity principle is advisable in general. But exceptions to this 
principle are sometimes recommended on the basis of professional intui- 
tions and research in psychology, which suggest that people like to per- 
ceive freedom and opportunity for self-expression when they answer ques- 
tions. Having only two alternatives sometimes makes them feel “hemmed 
in". They often prefer to indicate more details than a two-response 
question would allow. Thus, a certain artistic judgement must be exer- 


cised in determining which question type to use. 


The last paragraphs talked about why a two-alternative issue might be 
expanded into a multiple-choice format. Sometimes, there is a need for 
an opposite strategy: taking a multiple-choice issue and reducing it to 
a series of two-choice questions, a procedure which can be illustrated 
with the following multiple-choice question: 


Example 4.9 


"There has been a lot of discussion lately about the 
future of Quebec in Canada. Which of the following 


constitutional options do you prefer for the future 
of Quebec?" 


a. Quebec to become completely independent, both 
politically and economically. Oo 


b. Quebec to become politically independent but 
have an economic association with the rest 
of Canada. Oo 


c. Quebec to remain a province of Canada. O 


There are three options here which the researcher wants considered. But 
in fact, he is not tied to a multiple-choice format, for the question 
above can be easily broken into two two-choice questions as follows: 


Example 4.10 


1. "Which of the following constitutional options 
do you prefer for the future of Quebec?" 


a. Political independence for Quebec. (Go 
to Question 2). oO 


ee? Ae bi = 


b. Quebec to remain a province of Canada 
(Skip to Question 3). O 


2. You indicated that you prefer political inde- 
pendence for Quebec. Which of the following 
arrangements would you prefer to see? 


a. Quebec to become completely independent, 
economically as well as politically. QO 


b. Quebec to become politically independent, 
but have an economic association with the 
rest ‘of ‘Canada. O 


Again, then, we see that the researcher has an option about which question 
type to use, in order to gather data on the same issue. Clearly, breaking 
a multiple-choice question down into a series of two-way choices simpli- 
fies a complex decision. It helps the respondent to focus on details of 
the alternatives. 


But two cautions are in order regarding the disadvantages of breaking down 
multiple-choice questions in this way. The first concerns the increased 
length of the questionnaire. The second is rather more subtle, and con- 
cerns the researcher's assumptions in how he breaks down a multiple-choice 
question. He must test his assumptions and determine how a respondent 
would break down the decision in his own mind. Otherwise, there will be 
distortions in-the data. This claim can be illustrated with an example, 
as) presented next 


Consider a record distributor hoping to predict July sales. He may ask 
100 customers the following question and obtain the data indicated in 
brackets. "Which of the following record collections would you most 


prefer?" 


Example 4.11 


a. The complete set of Beethoven symphonies, 
played by the Toronto Symphony Orchestra. (25%) 


b. The complete set of Beethoven symphonies, 
played by the Winnipeg Symphony Orchestra. (25%) 


c. The complete set of Mozart symphonies i 
played by the Winnipeg Symphony Orchestra. (50%) 


On the basis of the data above, the distributor might order quantities 


in the proportion 25; 25; 50. 
Suppose he designs his questions otherwise, breaking them into two two- 
choice questions as follows: 


Se 


Example 4.12 


1. “Which of the following do you prefer, Beethoven 
symphonies or Mozart symphonies? 


( ) Beethoven (Go to Question 2) (50) 


( ) Mozart (Skip to Question Bi (50%) 


2. Suppose you could listen to Beethoven symphonies 
played by either the Toronto Symphony Orchestra 
or the Winnipeg Symphony Orchestra. Which would 
you prefer? 


( ) Toronto (50% of respondents who recieve the 
question, or 25% of total sample) 

( } Winnipeg (50% of respondents who receive the 
question, or 25% of total sample) 


On the basis of the data above, the distributor would again order quanti- 
ties in the proportion 25; 25; 50 for Beethoven/TSO, Beethoven/WSO, 
Mozart/WSO respectively. 


Note that in breaking down the question, the distributor has assumed that 
the respondent would use the following logic in making his decision: 
"Which composer do I preser? If Beethoven, which orchestra would I 
rather hear playing Beethoven?" In breaking down the original multiple- 
choice question, the distributor has predicted the respondent's logic, 
simplified the questionnaire and obtained the same data. 


But what if the distributor breaks down the original multiple-choice ques- 
tion as shown below? The data indicated beside each alternative are com- 


pletely plausible, and perhaps expected, given the pattern of responses 
hypothesized earlier. 


Example 4.13 


1. Which do you prefer, the Winnipeg Symphony 
Orchestra or the Toronto Symphony Orchestra? 


( ) Winnipeg (Go to Question 2) (50%) 

( ) Toronto (Skip to Question 3) (50%) 
2. Suppose you could listen to the Winnipeg 

Symphony Orchestra play Beethoven symphonies 


or Mozart symphonies. Which would you prefer? 


( ) Beethoven (50% of respondents who receive 
the question; or 25% of sample) 


( ) Mozart (50% of respondents who receive the 
question; or 25% of sample) 


es ae 


Note now that on the basis of the data above, the distributor's order would 


be in different Proportions, namely 50; 25; 25 for Beethoven/TSO, 
Beethoven/WSO, Mozart/WSO respectively. 


What has happened? Why do the results predict different sales? The 
distributor once again has simplified the questionnaire by predicting 

the respondent's logic, but he has predicted differently. He has assumed 
the respondent would break down his decision in this way: "Which orches- 
tra do I prefer to listen to? And if Winnipeg, which composer?" The 
Source of the different prediction is the assumption the respondents 
consider the orchestra-factor ahead of the composer-factor. 


In the context of this example, the danger seems minimal: it is "obvious" 
that composers are more central to a music-listener's preference for 
records than are orchestras. But in other circumstances, the hierarchy 
of importance of various factors may not be so obvious. 


The main point to be made is this: simplifying a multiple-choice question 
into a set of two-way choices involves assumptions about the respondent's 
decision-logic. If these assumptions are incorrect, the data may be a 
distortion of the "true" pattern of opinions. (The reader may recognize 
the similarity of this problem to a controversy surrounding Luce's 

"Axiom of Choice", as discussed in Debreu (1960).) 


This section, then, has demonstrated that the distinction between a two- 
choice and multiple-choice format is not a trivial one, based only on the 
number of alternatives relevant to the issue under study. There is a 
good deal of professional judgment which needs to be exercised in obtain- 
ing, simultaneously, the desired degree of simplicity, the respondent's 
cooperation, and valid data. 


4.2.3 The Checklist Question 


As illustrated above, a checklist question is appropriate when there is 
more than one alternative which may be "correct" for a given respondent. 
He is invited to indicate all the alternatives which apply to him. 


In asking whether a checklist format, as opposed to another format, is ap- 
propriate for his purposes, a researcher should be guided by the question 
on which he is seeking more statistical information. Since checklist data 
can be statistically analysed in several ways, there are quite a variety 
of research questions which the checklist can be used to address. Be Low 
are listed some of the indices which may be compiled from checklist data, 
and the research questions to which each index can be applied. Appropri- 
ate statistical tests are also named. 


Independent frequency counts for each alternative: 
How often is Alternative X chosen? Is it chosen 
by a majority of the population? Are there dif- 
ferences among subgroups of the population (for 
example, men versus women)? A chi-square statis- 
tic may be applied to the latter question, to 
investigate whether any observed differences are 


statistically significant. 


Within-list comparisons: How much more frequent “S 
the choice of Alternative X than of Alternative Y? 
Cochran's test (e.g., Hays, 1963) is appropriate 
here for statistical analysis. 


Within-list correlations: "When respondents 
choose Alternative X, how likely are they to also 
choose Alternative Y? Or, more generally, how of- 
ténemastherset ofalternatives” (&, ¥} 2.272) 
chosen as a package? 


Package size: What percentage of respondents check 
one only? What is the most frequent number of 
items chosen? (Such questions may be relevant, 
say, for tracking a publication distribution pro- 
gram, where one or more publications will be mail- 
ed to a respondent on request). Statistical com- 
parisons of average package sizes for various 
subpopulations could be done using a nonparametric 
analysis of variance. 


Most statistical analyses of checklist data require that certain assump- 
tions be met: 


that’ the alternatives are? distinc, “Otherwi serra 
respondent may use two categories to reflect the 

same response, and that response will be "double- 
counted"; 


that the alternatives are independent. The check- 
ing-off of one alternative should have no effect 
on whether or not some other alternative is 
chosen. (This assumption may be the basis for a 
Statistical test of "within-list correlations!’ , 
described earlier.); 


that all respondents perceive the same constraints 
in the number of allowable alternatives. (For ex- 
ample, in ordering free publications, some respon- 
dents may check off everything they want, while 
others stop at three because they think it would be 
unreasonable to expect more.) The meeting of this 
assumption is impossible to guarantee, but appro- 
priate question wording can help. 


4.2.4 Ranking Format 


The ranking format generates a more interesting discussion, for not only 
are there several ways to analyse ranking data, but, as we shall see, dif- 
ferent ways may give different indications of the favourability towards 
the alternatives. Thus, in analysing ranking data, it is particularly im- 
portant to be precise concerning what research question the analysis is 
intended to answer. Below are listed two possible indices for analysis. 


Pe 


Frequency counts of favoured alternative: For what 
percentage of people is Alternative X the favour- 


ite? 


How does Alternative X compare with others 


in this respect? 


Favourability judged by taking all rankings into 
account: What is the average rank that Alternative 


X obtained? 


How does Alternative X compare with 


others in this respect? 


To see how these two methods of analysis might yield conflicting indica- 
tions of favourability, consider the following voting procedure: 


Example 4.14 


Four candidates, Alice, Barbara, Charlotte and Dawn, 
are running for President of the High School student 
body. The voters are asked to rank order the candi- 
dates in order of preference. 


Alice and Dawn are the two strongest candidates, 
but they hold radically opposing views. Thus stu- 
dents who endorse Alice would strongly object to 
Dawn, and vice versa. Barbara and Charlotte are 
not really presidential material, but nice people 


whom no one 


objects to. 


In light of this situation, the following voting 
"data'' may be considered plausible. 


No. of 
A UL 


RANKS 

No. of No. of Nogsod 

li ES PAE UFR RS Average 
obtained obtained obtained obtained Rank 
Alice 225 0 0 175 Die 
Barbara 0) 400 0 0 260 
Charlotte 0 0 400 0 3.0 
Dawn MWS 0 0 225 ye | 


By the usual majority 


vote method, Alice would be President. But by the 


method of averaging ranks, Barbara would be President. In Fact, soanatt 
is an analogue of a true life story, a presidential vote in the author's 
own high school (several years ago). Because ‘the administration chose 
to average ranks, a candidate whom no-one wanted for president but to 
whom no-one had intense objections (as Barbara in our example) became 


president. 


This illustration points up a consideration which needs to be taken into 
account whenever ranking data are to be used as input to decision-making 


or policy formation. 


it be among candidates, 


The identification of the "people's choice" whether 
potential product-lines, or constitutional options 


ta are analysed. 
for the country's future, depends on how the data y 


A particular word of caution needs to be added regarding the averaging 
of ranks. This manipulation of the data relies on an assumption, an as-~ 
sumption which must be true to ensure valid interpretation of the data. 
The assumption is that the ranks are on an "interval scale ine. that 
the interval between rank 1 and 2 is perceived to be the same distance 
as the interval between rank 2 and 3, 3 and 4 and so on. Only this 
equivalence of distance between ranks makes valid the arithmetic manipu- 
lations involved in averaging. 


Here are some sample reactions on the part of the respondent which should 
alert the researcher that the assumption does not hold: "Well, I've mana- 
ged to rank the first five, although they're all pretty good. But this 
sixth one is inferior by far to any of the others." (Distance betwen 
rank 5 and 6 is greater than between other ranks.) "After indicating my 
first three choices in order, I can't really rank the fourth, fifth and 
sixth - I wouldn't want any of them.'"' (This is a reaction often obtained 
in the author's experience. Ranks 4 to 6 are not meaningful to the res- 
pondent. Or, at best, distances among ranks 4 to 6 would be smaller than 
among ranks 1 to 3.) ‘My first choice is a toss-upebétween*Asand B: “OK, 
I'll put A first B second." (Distance between A and B obviously minimal, 
yet in averaging ranks, it would’ be treated as equivalent to distance 
between B and the alternative ranked third.) 


When there is doubt about whether the equal-distance assumption would 
hold in a given context, three courses of action are possible: 


Stick to frequency counts only. Perform no 
adding or multiplying operations on the 
numerical ranks. 


Explicitly mention in the instructions that 
respondent should assign ranks so that the 

distance between ranks is equal everywhere. 
Have him note when this cannot be done, and 
omit) those eases in analysis of data 


Change to rating scale format if feasible. 


Details for analysing data with a rating 
format are given in the next section. 


4.2.5 Rating Scales 


In a question involving a rating scale, a respondent is explicitiy asked 
to represent his opinion with a numerical value. The numerical values 
so obtained act as direct input to a statistical analysis, an "analysis 
of variance” for example. An analysis of variance would determine 
whether there are differences in the mean ratings among pre-selected 
subsets of the respondent population. 


The assumptions necessary for valid rating-scale data are chiefly as- 
sumptions about the respondent: He must understand the rather abstract 
task of representing his opinion by a mark on a line; he must treat the 
scale in the way intended by the researcher, perceiving psychologically 


ey a 


equal intervals where physically equal intervals appear on the page. 
Finally, most of the more sophisticated statistical tests allowed by 


rating scales include the assumption that the numerical responses will 
be normally distributed. 


This completes the review of indicators which can be obtained and hypo- 
theses which can be tested by means of different question types. Clear- 
ly, it is such indicators and hypotheses which should be identified 
first in a survey plan, and which should guide the selection of the 
types of questions suited to the purpose. 


4.3 Repeating Question Types to Build Attitude Scales 


In this section, it is intended to show that the typology presented 
earlier provides one with all the raw material necessary to demystify 
the building of attitude scales. Five well-known attitude scaling tech- 
niques will be shown to be nothing more than a composition of questions 
selected from each of the five types described above. 


4.3.1 Thurstone Scale: A Composition of Two-Choice Questions 


In a Thurstone scaling procedure, a subject is presented with a list of 
statements, each of which he is asked to either endorsé or reject. Thus, 
a Thurstone scale is simply a collection of two-choice questions. An 
example follows, part of an attitude scale consisting of statements 
about war (adapted from Thurstone, 1932). 


Example 4.15 


Instructions: For each of the following statements, 
please indicate with a checkmark whether you agree 
with+1t- or? disagree’ with vt. 


Agree Disagree 


ieeWar “re "an fubive “strueete 
resulting in self-destruction. oO 0 


2. We want no more war if it can 
be avoided without dishonour. oO 0 


3. War is sometimes necessary 
because right is more important 


than peace. Oo eI 


4. War is glorious. O U 

As shown in the example above, statements are selected to represent sever- 
al different "positions" on the issue in question. The shaetaiiacias= vary 
according to how strongly they apeak for or against the ered kel Mees 
statement such as "War is glorious reflects a duet Sheed: whic : abl tear ids 
and favours it strongly, while a statement such as "War is a futile strug 


gle resulting in self-destruction 
someone strongly opposed to war. 


'' would probably be endorsed only by 
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Where do statements originate which constitute such a scale? The proce- 
dures outlined originally by Thurstone and Chave (1929) advised the 
collection of a hundred or more statements on the topic in question. 
Statements could be gathered from anywhere: radio, newspaper, political 
speeches, etc. Criteria were specified by Thurstone and Chave for 
editing the pool of statements into a reasonably-sized package of 

"good" items. 


The authors were quite explicit in their criteria for a good item. Each 
statement, they said, should be clear, brief and straightforward. It 
should be relevant to a respondent's overall attitude to the issue in 
question. It should be unambiguous. And finally, as a package, the 
statements should represent the whole range of opinions possible about the 
attitude variable. The package of statements was called an attitude scale. 


Statistical techniques were specified by Thurstone and Chave to assess the 
relevance and ambiguity of items, as well as to translate responses into 

an attitudinal score. The "output" of a Thurstone scaling procedure is a 
numerical score for each respondent, representing the relative strength of 
his attitude for or against the issue in question. This score has no mean- 
ing by itself; it gains meaning wnen compared with scores of other indivi- 
duels. Average scores of subgroups in a population can be compared: In 

the example above, we might expect to find a difference between those who 
have fought in a war and those who have not. 


The details of how one translates responses into scores can be found in 
Oppenheim (1966). Areas of application of Thurstone scaling have included 
attitudes toward ethnic and cultural groups (MacCrone, 1937; Hinckley, 
1932; Eysenck and Crown, 1949), social institutions (Remmers, 1943), war 
(Dudycha, 1943; Ferguson, 1935), political candiates (Beyle, 1932), and 
religion (Granneberg, 1955). 


4.3.2 Likert Scale: A Composition of Multiple-Choice Questions 


Like a Thurstone scale, a Likert scale is a collection of statements. The 


respondent considers each statement and reports how closely it reflects 
his own opinion. 


The chief difference between Thurstone and Likert procedures lies in the 
number of response alternatives for each statement. For Thustone scaling, 
there are two; for Likert scaling there are at least five. The respondent 
indicates not only whether he agrees or disagress, but how much he agrees 
or disagrees. An example follows, a Likert scale based on statements 


about the glorification of sports (adapted from Jones' (1972) study of 
athletes). 
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Example 4.16 


Strongly Strongly 
Agree Agree Neutral Disagree Disagree 
It's not how you play 
the game, it's whether 
you win or lose QO O oO O 2! 
An athlete igs a gen- 
tlemen who represents 
his school Oo Oo Oo O Oo 
Champions come in all 
sizes O O oO oO oO 
Winning isn't every- 
thing, it's the only 
thing O 0 0 QO O 


"Agreement" is not the only response dimension one can use. For suitable 
issues, degrees of "approval" may be used. Degrees of frequency are 
another possibility. 


As with Thurstone's procedures, the construction of a Likert scale begins 
with a collection of a large number of relevant statements from any 
source. Editing takes place next, according to criteria which are much 
the same as Thurstone's. Items should be concise, straightforward and 
unambiguous. They should all be relevant to a single attitudinal vari- 
able. And again, statistical techniques exist to assess the relevance 
and ambiguity of items, so that unuseful items may be omitted. 


The output of a Likert scale is a numerical score, compiled from all the 
statements. The score has its use only in a relative sense, when compar- 
ed to scores of other individuals in a population. The reader may con- 
sult Likert (1932) for the original outline of the procedures, with more 
recent summaries being found in Oppenheim (1971) Boyd, Westfall and 
Stasch (1977) and Scott (1968). 


4.3.3 Guttman Scale: A Special Case of the Checklist Question 


A Guttman scale is a checklist question with special properties. Each 
of the items represents an increasingly strong expression of a single 
attitude. The most well-known example of a Guttman scale was used in 
an attempt to measure peoples’ attitudes toward racial groups, as shown 


below. 
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Example 4.17 


Into which of the following relationships would 
you be willing to accept a member of the 
ethnic group? 


Close kinship by marriage 

Personal chums in the same club 

Neighbours on the same street 

Employment in same occupation 

Citizenship in the country 

Visitor to country 

None of the above. Exclude members of that 
group from the country. 


(2) 43) Jee) fer eles, 


The aspect of interest is clearly in the respondent's cutoff level. 
Because the items are ordered according to a decrease along a variable, 
we would expect a respondent to check all items below his/her cutoff level. 


Such an ideal list of items is not’ always possible, and there are statis- 
tical techniques to assess whether the scale is close enough to ideal to 
be considered useful. A respondent's answers to Guttman type questions 
result in a single score assigned to that respondent. The score will es- 
sentially reflect the so-called “cutoff-point" referred to earlier. 

Again the score is meaningful only in a comparative sense. 


The researcher may consult Guttman (1950) for the complete theory and 
scaling procedures which characterize the technique, and will” find” setee— 


ted applications in Podel and Perkins (1957), Wallin (1953) and Clark and 
Kreidt (1948). 


4.3.4 Conjoint Measurement: A Composition of Ranking Questions 


Conjoint measurement is an attitude scaling technique whose potential is 
even yet being explored. The technique relies on the respondent's abili- 
ty to rank-order a set of alternatives, and from this rank ordering 


attempts to assign quite precise measures of "value'’ to each alternative. 
Let us begin with an example. 


Example 4.18 


Suppose you were in the market for a house, and the 
three houses described below were available. Based 
on these very brief descriptions, please indicate 
which house you would prefer most, second most and 
third most. Answer by putting a "1" beneath the 
description of your most preferred house, a "2" 
beneath the house preferred second most, and so on. 


House A 


Room sizes: 
generally large 


Number of rooms: 


r. 


ee 1) 


House B 


Room sizes: 
generally small 


Number of rooms: 


House C 


Room sizes: 
generally large 


Number of rooms: 9 


Price $60,000 Price: $70,000 Price: $85,000 
Location: City Location: Location: Outside 
Suburbs Downtown City limits 


NOUN wank fi) Your rank () Your rank “ew) 
After respondents make several such choices in which the "levels" of 
several factors (room size, price, etc.) are Varied; iamstatistical 


analysis is undertaken with the following outputs: 


the relative importance of the different factors 


(room size, price, etc.) in respondents' overall 
choices; 


the relative value of individual levels of each 
factor (for example, how much large rooms are 
"worth" in comparison to small rooms); 


an overall numerical value for each possible 
choice (House A, B, C), reflecting its favour- 
abality. relative to, the others 


The evident advantage of the conjoint measurement technique is that it 
simulates lifelike situations in which consumers consider benefits in 
combination with each other - often having to give up some of one benefit 
in order to get more of another. 


Detailed analytic procedures for conjoint measurement are set out in 
Green and Rao (1971) and Johnson (1974). 


4.3.5 The Semantic Differential: A Composition of Rating Questions 


The semantic differential method of scaling attitudes involves presenting 
the respondent with a concept (278.5 the name of the attitudinal object) 
followed by rating scales along many dimensions. Each rating sore: is 
"anchored" by a bipolar adjective, such as cool and hot, mild ¢ datos, 
and so on. The following example should clarify the nature of the task: 
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Example 4.19 


QDW Cigarettes 


Rating 

1 2 3 4 5 6 rs 
Cool = aol mo = a ies . Syetice 
Thick -a'¢ ie —_ a. nS we thin 
Mild en rr - a et 28 _... .strong 
Masculine —__ kes eae —_ —. _ _» Feninane 
Savory on poe ae, Hh Me at J, Jdastetess 
Unusual Usual 


The output of a semantic differential task is not one score for each res- 
pondent, but several. First, there is a score out of 7 for him along each 
dimension. But statistical analysis of his several ratings result in a 
more compact set of scores, a set of scores which isolates the most impor- 
tant variables among all the ratings. This "compacting" of data is done 
by means of a factor analysis. 


Semantic differentials are valuable when a respondent's attitude is ex- 
pected to be teo "complex" for measuring on a single dimension, and when 
comparisons among several objects are desired. By choosing bipolar ad- 
jectives judiciously, one can obtain insights into how specific feelings 
and perceptions contribute to the overall emotional appeal of a given 
object. 


This completes the review of selected attitude measurement techniques. 

They have been reviewed in a way which emphasizes their construction 

rather than their underlying theory. The interested reader will find 
dozens of books which do describe the underlying theory. But, hopefully, 
the practitioner has been convinced in this section of the mechanical 
straightforwardness in construction of basic attitude scales. He can begin 
to exploit the many years of theory and development which make attitude 
scales accessible and useful for practical research. 


4.4 Open-Ended Questions 


This section follows much the organization of the section on closed-ended 
questions. It begins with discussions of sub-classifications and types 
of analyses. In fact, though, these discussions serve to highlight the 


difficulties of any sub-classifications or analyses, and thus point to 
obvious disadvantages of open-enders. 
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4.4.1 The Difficulty of Sub-Classification 
ee) VE UD TU Lass Tication 


Payne (1951) made some attempt to delineate the variations of open-res- 
ponse questions which could be asked. There are, for example, lead-in 
questions ("How do you feel about the present bus service?"), requests 
for suggestions ("What could the company do to build better relations 
with the public?"), taps of knowledge or memory ("What have you seen or 
read about the new metric measurement system?'') and probes ("Are there 
any other things you can remember? What things?"). But it is not ob- 
vious whether his classification scheme is complete, nor has it seemed 
to prove useful to students of questionnaire design since. 


4.2.2 The Difficulty of Analysis 


How to analyse, statistically, the responses to open-ended questions is 
another topic which defies reduction to a set of systematic principles. 
Some form of "content analysis" may be performed, in which the respon- 

dent's communication is subjected to codification and codified data 

to standard statistical analyses. That is, we have evolved techniques 

which permit the application of the scientific method to the fuzzy area 
of social communication. 


But it is precisely these laudable attempts at rigorous analysis which 
highlight the limitations of the open response question. Let us look at 
the coding and analysis procedures, and consider what compromises are 
made, what assumptions necessary for certain analyses are overlooked, in 
the determined application of statistical methods to open-ended 
responses. 


The objective of coding procedures for open-ended questions is to devise 
a set of response categories which "capture" most respondent answers. 
After devising such categories, the researcher can take each respon- 
dent's answers and re-record them in a closed-ended format. 


How then does the researcher devise a set of response categories for a 
given question? He may begin by reading a randomly selected subset of 
all returned questionnaires, and listing all responses given to a par- 
ticular question. Through trial and error, he groups responses under 
categories, choosing category names which suit the intended meanings of 


a group of responses. 


Once responses are categorized, they can be submitted to standard sta- 
tistical analyses used for closed-ended questions. It is at this point, 
ating open-ended data as though they came from a closed-ended 
that major statistical assumptions are made. Principal among 
these is the assumption that all respondents have answered the pane AP 
plicit closed-ended question; they have all, presumably, had mental ac 
cess to the categories devised post hoc by the researcher. 


tn Ere 
question, 


for many of the statistical manipulations typically 

‘ed out, is that each respondent's answer is equal in weight to each 
ea ‘ We sume that we are sampling comparable amounts of their 
fare a [eee PER But the assumption is simply not warranted, 
minds 


Another assumption, 


Bi 


if only for our very strong intuitions that reticent: people are mat 
necessarily less detailed in their internal beliefs, or that garrulous 
people often say superfluous things that they have not at all thought 
out. Obtaining an open-ended response is very much like a sampling ex- 
ercise. And variations in people's verbosity implies that we have un- 
equal-sized samples, an indeterminate proportion of each respondent's 
population of thoughts. 


A specific illustration of the potential bias inherent in the equal 
weighting of respondent's answers is given in Boyd, Westa and Stasch 
(1977, p. 234). The authors claim that it is the upper-income, better 
educated segment of the population who, being more articulate, are like- 
ly to generate more points in answering open questions. This group, for 
example, might mention three reasons for liking '"X" soap on the average, 
while the average for others might be one. Then, if 100 people from the 
articulate group and 100 others are interviewed, 400 reasons for using 
"%' soap will be recorded. But three out of every four/of ithe reasons 
will have been given by the articulate group. If the two groups differ 
in attitude, then a statistical summary containing "number of mentions" 
for each of the various recorded reasons will be biased. 


The interviewer, too, can encourage a similar bias in the data: in con- 
ducting an interview, he may vary markedly from other interviewers in his 
willingness to probe for more information. 


To overcome the problems of unequal length answers, the data analyst may 
select only the firstspoint, say, made by each respondents This tigutaneag 
mount to the rather questionable assumption that the respondent's first 
point is the one he considers most important, the one he would select if 
he were answering a multiple-choice question. A similar argument applies 


even if the respondent's first two or three points are included for 
analysis. 


Put another way, we have no way of controlling the selectivity of the 
respondent's information. Are the points he mentions, as he talks away, 
the points which are really most important to him, or the points which 
come to mind most easily? Are they ideas from a newspaper which he has 
just put aside for this survey interview, or are they entrenched values? 


Finally, the problem of "information selectivity" may be generated by the 
interviewer too. In the onerous task of recording answers verbatim he may 
highlight those points he believes to be more interesting or relevant. 


These are all factors which tug uncomfortably at the rigorous treatment of 
data which content analysis promises. Simply put, it is hard to tell 
whether the demands on time and energy entailed by open response questions 
are worth it. Certainly, though, the last word cannot be said until re- 
Searchers reach the limits of their ongoing efforts to improve the quality 
and meaningfulness of such data, through improvements in methodology, 


interviewer training, and through more in-depth analysis of human 
communication. 
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4.4.3 Advantages of Open-Ended Questions 


Why, in the light of all that has been said above, does the practical 
world maintain open-ended questions? There are certainly identifiable 


advantages to open-ended questions, characteristics tied perhaps to their 
qualitative rather than quantitative use. 


In a focus group interview, or other preliminary exploratory study, they 
are useful in generating hypotheses, in giving the researcher a more ob- 
jective view (than his own) of how to phrase hypotheses with maximum ef- 
ficiency. He ican then systematically produce structured items on a 
subsequent questionnaire which will yield answers directly tied to his 
hypotheses. Moreover, contributions from many individuals help the 
researcher determine a reasonable range of responses for a structured mul- 
tiple choice item. In summary, open-ended questions are at least as use- 
ful as other descriptive sources of information which are tapped in the 
preparatory stages of questionnaire design. 


But they may play a role, too, in the "real" questionnaire. They are ad- 
visable when it is expected that some respondents may have no information 
Or no opinion about an issue in question: structured items might induce a 
response which reflects falsely on the respondent's knowledge or attitude. 
Side comments and explanations which respondents make often assist in 
interpreting data. And, too, there is the mere aesthetic value of open- 
response data as a source of "quotable quotes", lending colour and authen- 
ticitystolalifiinal: report: 


On the point of their artistic value, it is possible to add one more argu- 
ment, less well-formulated, but no less important. The point is princi- 
pally relevant to the interview situation. Open response questions 
imitate the everyday process of human communication; consequently they may 
be used with all the subtle tact, the insightful probing, the control of 
tension that we use when we cajole, negotiate or share thoughts in real 
life. They may be used to warm-up a respondent at the beginning of an 
interview, to introduce him to the topic, to put him at ease. They give 
the respondent an opportunity for self expression and ‘werbad "catharsis %, 
which act as incentives for his continued participation. After a stacca- 
to series of closed-ended questions, an open-ender may lend a pleasant 
change of pace. It may smooth out the frustration of being forced to 
choose between two unsatisfactory alternatives without an explanation. 
(Even some courts now allow minor traffic violators to) plead "guilty, 

with an explanation".) Open-ended question may help to gently bridge a 
gap between two different topic areas in a questionnaire. 


In short, an interview is a delicate communication between two human 

beings and open-ended questions may be artfully used to make that commu- 
, . * 

nication as supportive and effective as possible. 


4-5" Summary 


y is clearly subsidiary to the mainstream directions of 
But it facilitates such research, by highlighting cur- 
. Like a toolbox, a question typology 


A question typolog 
survey research. ' | 
rent capabilities of questionnaires. 
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makes a researcher aware of what he can build, and may suggest creative 
ideas. For example, the introduction of conjoint measurement, it 4s oreen 
acknowledged, allowed researchers to explore new kinds of hypotheses; it 
proposed a new style of questioning which permitted a more sophisticated 
analysis of consumer preferences. 


The typology heirarchy is dominated by the distinction between closed-ended 
and open-ended questions. Most published discussions are organized around 
a comparison of advantages and disadvantages of these two main types. The 
present discussion has reluctantly referred to the commonly acknowledged 
"advantages" and "disadvantages" but has attempted to establish a quite 
different focus. The intended focus has been on the use of the data, and 
on the different uses to which each question type is more appropriately 
suited. That is, the disadvantages of open-ended questions imply that they 
are simply less appropriate in some circumstances. For different circum- 
stances, they are ideally suited to a particular use. 


Putting a question typology into a context of research objectives and data 
use does much it seems, to reinforce the proper role of methodology: the 
role of a support mechanism for the collection of valid and reliable and 
usable information on any substantive topic. 
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WORDING 


There are a number of considerations that should be kept in mind when 
developing the wording of questionnaires. These considerations will help 
development of a good questionnaire according to such serhteniasasche 
ease of comprehension of the questions by both respondents and interview- 
ers, the precision of meaning of questions, the flow of thought through 
the questionnaire, the ease of data processing and the meaningfulness of 
results. 


Following is a selection of suggestions for consideration when developing 
the wording of a questionnaire. Each must be examined from the point of 
view of the survey subject matter, the survey objectives and the uses 
that will be made of the results. Some of these approaches will seem 
appropriate by intuition, others will draw their support for use from 
empirical evidence. 


5.1 Statement of Alternative Answers 


The respondent's train of thought is led by the choices offered in a 
question. If a question contains a selection of answers from which the 
respondent is to choose, it is best to present all possible answers and 
leave no alternative implied. If for example only one of two possible 
answers is presented, respondents are more likely to agree with the ans- 
wer present than to formulate the alternative in their minds and to 
choose objectively between the two. The following example presents two 
wordings of a question and the corresponding results when asked of two 
matched groups. Note that the first example presents only one possibi- 
lity and leaves the alternative, that companies could avoid layoffs, to 
the respondent to formulate. 


Example. 5.1 


Question: 


(1) Do you think most manufacturing companies that lay off workers during 


slack periods could arrange things to avoid layoffs and give steady 
work right through the year? 


Results: 


63% said companies could avoid layoffs; 
22% said they couldn't; and 
15% had no opinion. 


The question was then reworded to state the implied alternative that lay- 
offs are unavoidable. 


Question: 


(1) Do you think most manufacturing companies that lay off workers in 
slack periods could avoid layoffs and provide steady work right 
through the year, or do you think layoffs are unavoidable? 


05 Oe 


Results: 


35% Said companies could avoid layoffs; 
41% said layoffs are unavoidable; and 
24% expressed no choice. 


About two thirds of respondents agreed with the idea presented in the 
PEGS U question. However, when the alternative was explicitly presented, 
the proportion agreeing with the initial idea was considerably lower. 


The problem of motivating the respondent to formulate alternatives is 
magnified when there is more than one alternative to be considered. The 
following example illustrates the distribution of responses to two ques- 
tions, the first one with limited alternatives presented in the question 
and the second one with all alternatives presented. The two questions 
were asked of the same group at two points in time. Note that the com- 
plex nature of the question presenting all alternatives required a visual 
aid. The responses were listed on a card which was handed to the respon- 
dent when the question was asked. 


Example, 5.2 
Question: 
Suppose they do set up a plan to provide workers with unemployment and 


health benefits through royalty payments. Who should manage the fund: 
the companies, the government or the union? 


Results: 
Government 33% 
Union 18% 
Company 18% 
Company and Union 7h 
Company and Government aly s 15% 
Union and Government 1 . 
All three 6% 
No Opinion 16% 
Question: 


Which of these should manage the fund? 
(present card) 


Government 

Union 

Companies 

Companies and Unions 

Companies and Government 

Unions and Government 
Government, Unions and Companies 


aod He 


Results: 
Government 18% 
Union 4% 
Companies 13% 
Companies and Unions 18% 
Companies and Government 8% 52% 
Unions and Government Sy r 
Government, Unions and Companies 21% 
No Opinion 13% 


When the possibility of combinations wasn't offered, only 15% went beyond 
the question's limited choice and volunteered combinations. Later, when 
all the combinations were explicit and presented visually, about one half 
of the respondents chose from among the combinations. It is possible that 
the shift reflects changes in opinion that occurred from one survey to the 
next. However, the researcher (S. Payne) felt 'it seems more likely that 
the difference in results arises from the explicit mention of the combina- 
tions in the second survey = Payne refers to this as the 'pie a la 


mode' problem. If you offer guests pie or ice cream, few will ask for 
both. 


Response categories for multiple-choice questions should: 


(a) list all possibilities (including a non-response or not- 
applicable category where appropriate); and, 


(b) not overlap (i.e., they should be mutually exclusive). 


Where all possible answers are not known ahead of time, or cannot all be 
listed because there are far too many, a grab-bag "other" category should 
be included. Often this is: "Other (please specify)" with enough room for 
the specific answers to be written in. There might be an important cate- 


gory not dealt with, that can be detected from analysis of the "other" 
answers. 


Often, a pre-test of the questionnaire can be useful, to determine the 
appropriateness of answer categories as perceived by respondents, as well 
as to see if there is any pattern of answers in the "other" category that 
deserve listing as a separate category (either because they are mentioned 
at least as much as a category already listed, or because there might be 


too many or too high a proportion of answers ending up in the "other" 
category). 


Here are some illustrations of the kinds of problems that can occur if the 
categories are*not mutually exclusive and exhaustive. 


Example 5.3 


What is your age? 20-30 O 
(Check appropriate 30-40 Oo 
box) 40-50 Oo 

50 and over O 


- 41 - 


In which category are the responses of those who say they're exactly 30 
years old to be recorded? With two categories including '30', responses 
will be allocated in some unknown way according to the whim of each in- 
terviewer or respondent. How about a respondent who is under 20 or 
refuses to give his age? There is no place to record these answers. One 
solution is to provide for all the categories, and to avoid overlapping, 
by offering categories such as: under 29),.20-29 9 30-39. 40-49  .S0eand 
over, not stated, not known and refused. 


There are instances where the categories presented are mutually exclusive 
in the literal sense yet, due to varied interpretations of the terms used 
in everyday experiences, the categories appear to overlap. In the fol- 
lowing example, disposition of waste via decontamination in the scienti- 
fic sense may describe a process which is completely different from the 
process of disposition of waste via neutralization. Yet, in the working 
world, some companies may use the two terms interchangeably and they 
would be confused when asked to make a choice between the two in a 
questionnaire. 


a pe 


In what manner is a chemical discharged by your plant presently treated 
or disposed of? (Check one or more) 


sewer ( 
landfill ( 
tailings ( 
decontamination ( 
neutralization ( 
ion exchange ( 
carbon absorption ( 
storage ( 
other (please specify) ( 
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It is, therefore, important to ensure that the maximum number of respon- 
dents understands each category so that there is no overlapping and am- 
biguity that could lead to overlapping. A pretest in which respondents 
are asked directly whether the items presented do overlap would be a use- 
ful means of resolving this problem. 


5 et Order. Bias 


When respondents are asked to choose among a number of alternatives, a 
rge that is the result of the order in which 


attern of answers may eme 
: as well as reactions to the ideas themselves. 


the ideas were presented, 


Consider the following example (Payne, page 84). Several ideas, among 
which were ideas A, B, C, and D, were presented in different orders, to 


matching samples of respondents. In each case, the ideas A, B, fa 
were selected more often when they were at extremes 1n the lists than 


when they appeared near the centres. 


ney ee 


Examp be 575 


Idea A was selected by: 


27% when it appeared at the top of the list; 
17% when it appeared near the centre; and 
23% when it appeared at the bottom of the list. 


Idea B was selected by: 


11% when at the top; 
7% when near the centre; and 
7% when at the bottom. 


Idea C was selected by: 


24% when at the top; 
20% when near the centre; and 
21% when at the bottom. 


Idea D was selected by: 


23% when at the top; 
16% when near the centre; and 
18% when at the bottom. 


This sort of situation is quite common. Some respondents will hang onto 
the first idea of a series, when it becomes just too much to remember 
them all. Others tend to remember the last one, since nothing else comes 


along to displace it from their attention. The ideas in the middle tend 
to lose out, either way. 


One solution to the problem is to present the ideas in all possible 
combinations, to as many randomly picked groups of respondents as there 
are orders of the ideas. This will help balance out some of the bias 
due to ordering. Of course, this can cause practical problems, such as 
requiring more than one version of the questionnaire, with consequent 
control problems in the fieldwork. It can also mean far more alterna- 
tive versions - with three ideas, there are six possible combinations; 
with four ideas, there are 24, and so on. However, with some compromis- 
ing and careful ordering, a practical smaller number can usually be 
found. Here is an example of how one question with six choices and a 
possible 720 orderings, was handled (Payne, page 85). 
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Example 5.6 


In which of these industries would 


you say there is the greatest competi- 
tion among companies? 


Card A Card B Caran 
Automobile Steel Oil 

Oil Automobile Steel 
Railroad Goal Chemical 
Steel Oil Automobile 
Coal Chemical Railroad 
Chemical Railroad Coal 


In this selection, each industry appears once at an extreme end of the 
list (either first or last), once in the middle (3rd or 4th), and once in 
2nd or 5th position. As well, no two industries appear together twice. 


5.3 Other Biases 


There are a number of other sources of bias which will affect survey re- 
sults. One of these sources of bias is the tendency of respondents to 
lean towards favourable and socially acceptable answers and towards ans- 
wers that are most favourable to their self-esteem. 


If there is an agree-disagree scale, with an equal number of intervals on 
each side, the average of all answers will tend to be toward the ‘agree’ 
side of the mid-point. If open-ended liking and disliking questions are 
asked, as a rule-of-thunb about twice as many liking responses as dislik- 
ing will usually be offered by respondents. This is a very pervasive 
pattern that becomes apparent once results of a small number of surveys 
have been analyzed. Oppenheim has described this as an ‘aquiescence 
bias’ (" ... a general tendency towards assent rather than dissent..."). 
Even in circumstances where respondents might be expected to be strongly 
opposed to something or someone, there still emerges a leaning towards 
the favourable end of the scale, serving to soften the strength of nega- 


tive attitudes expressed. 


To some extent, the effect can be decreased ina scale question by care- 
ful selection of scale wordings, but it can never be eliminated. For ex- 
ample, a scale should always include positive and hegalive: Rtalements:, 
although there are no firm rules based on empirical evidence suggesting 
just what words, how large the apparent intervals should be in the scale, 
whether there needs to be an equal number of positive and negative state- 
ments, or whether an apparently neutral mid-point is needed. Some scales 
such as Likert's 5-point scale (strongly agree, agree, art & sfc ROE 
disagree, disagree, strongly disagree), appear to work satis A oe hie 
and have the sanction of extensive use. The literature ina pir std 
their development is immense, and it is not our pane er is im 
book to cover the topic thoroughly. Some references for futher reading 


are given at the end of this section. 


a 


Given the tendency ttowards positive bias in attitudinal responses, results 
should not be accepted as showing an "absolute" level of agreement, but 
be interpreted relatively, e.g., in comparisons between age or sex or 
region classes or with results from another survey, as showing greater or 
lesser agreement. 


As mentioned earlier, respondents will tend to choose an answer that is 
most favourable to their self-esteem. They will also tend to choose ans- 
wers that they think make them look intelligent or thoughtful to an inter- 
viewer. As well, answers that they think the interviewer would like them 
to give or that they think are in accord with social norms, may be offer- 
ed. A further factor leading to the bias is a desire to be polite to an 
interviewer, who is most often a stranger. In being polite, respondents 
will hesitate to say unkind things. In self-enumeration surveys the ef- 
fect of this bias may mean a greater non-response among respondents who 
feel that their true answers might not be socially acceptable. The effect 
can be reduced although never entirely eliminated, by minimizing the de- 
sirability of one response over any other. 


Here is an example of a question which may entail "social desirability". 


Example 5./7 


Did you vote in the last election? 


Some respondents may think "it's my duty as a good citizen to vote in 
elections or at: least 1, think that's thessocial norm,.-.soul hadsbeteer 
indicate to the interviewer that I agree with, and follow, the norm". 


An alternate wording would be: 
Were you able to get to the polls to vote in the last election? 


This wording implies as an acceptable answer that not everyone did make it 
to the polls, and maybe the respondent didn't either. Since it is best to 
leave no alternative implied, this wording could be further improved by 
making the position explicit. 


Many people did not have a chance to get to the polls to vote in the last 
election. Were you able to get there? 


One of the easiest traps the survey designer can fall into (often uncons- 
ciously) is that of using leading or loaded words. These words may sug- 
gest an answer to the respondent which he thinks the designer wants or 
may make it easier or more desirable to answer in one particular way. 
Leading or loaded questions can result from the use of emotionally char- 
ged wording, the use of stereotyping, reference to the status quo, incom- 
plete presentation of alternatives, reference to items which touch mat- 
ters of prestige and pride, and personalization of questions. 


Here are some rather extreme and even ludicrous examples. 
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Example 5.8 


(a) Does small business need a government wet nurse in all its daily 
activities? ("No" response - 97%) (Payne, page 180). 


(b) Do you think that Michigan should enact Gestapo-type laws that would 
reward citizens for spying on and reporting on other citizens for 


such offenses as air or water pollution? 


(c) Should Canada ignore the principles of the Bible, which is the law of 
God and morality, and legalize murder by abortion? 


(d) How do you generally spend your free time - watching television, or 
what? 


(e) How many times have you damaged your body by taking LSD? 


(£) Do you have respect for the New Metro council members - such as 
Fred Billings? 


Sometimes the lead can be subtle. Here are two wordings, both using very 
strong active words to describe a government control. See how the results 
were influenced by the wordings: (Payne, page 57) 


Example 5.9 


Do you think the United States should allow public speeches against 
democracy? 


Results: 
Should allow 24 
Should not allow 62% 
No opinion 17% 


Do you think the U.S. should forbid public speeches against democrary? 


Results: 
Should not forbid 39% 
Should forbid 46% 
No opinion 15% 
Apparently, while 62% would not allow public speeches in response to the 


respondents bristled at the idea of anybody forbidding 


i tion i 
7S le so that only 46% would forbid them in response to 


them to do something, 
the second question. 


Here is another example of how a subtle change of wording in presenting 
a concept may lead to a different conclusion: (Payne, page 65) 
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Example 5.10 


Would you favor adding a law to the Constitution to prevent any President 
of the United States from serving a third term? 


Would you favor changing the Constitution to prevent any President ‘of the 
United States from serving a third term? 


Results: 


Adding Changing the 
a law Constitution 


Yes 36% 20% 
No 50% 65% 
No opinion 14% 9% 


The analysts who worked on these results hypothesized that to many res- 
pondents the United States Constitution is regarded as something not to 
be tampered with. About 1/3 would accept adding a law but only 1/4 would 


agree to "changing" the Constitution, yet both alternatives were designed to 
describe the same process: 


5.4 Choice of Words 


The language of a question should be simple. Words used should be fami- 
liar to and appropriate for the survey population. The kind of simple 
vigorous language generally used by newspapers could be worth aiming for. 
It is important to choose these words carefully since simple words can be 


vague or ambiguous. A trade-off may be required betwen accuracy and 
length. 


Here are some examples whichviolate this principle by their lack of 
accuracy. 


Example 5.11 


What is your income? 


Does "your" refer to the respondent personally, or to his family or 
household? What time-period should he think of - last week? last month? 
the last 12 months before interviewing? last calendar year? last fiscal 
year? What is to be included in "income" besides salary or wages - tips? 


piece-work pay? income from other sources, such as family allowances, 
bank interest, the sale of a car or a house? 


Example 5.12 


How much sugar do you usually consume? 


Par 


Should the respondent answer in ounces? pounds? kilograms? for what time- 
t 

period? If he "usually" consumes lots one week, little the next, what 

should he answer for "usually"? Selection of a reference period for 


"usually" if not specified, will probably vary from respondent to respon- 
dent, and the analyst won't know what it is. 


Example 5.13 


What proportion of your production is sold to customers: A? B? C? 


What time-period should the respondent think of, in extracting or compil- 
ing production totals from company accounts or records? "Production" 
Should be clearly defined, as the research may or may not want to include 
for example, production that is spoilt, otherwise not suitable for sale, 
or that may be given away in sales promotion work. Ambiguities may arise 
from the word "sold", and in references to "customers". Companies may 
use varying accounting definitions as to what constitutes a purchase. Is 
a shipment out of the company's premises a "sale"? What about returns 
for credit - are they taken into account in measuring "sales"? 


> 


Example 5.14(1) 

(Respondent is shown a bottle of orange-drink. ) 
How much orange juice do you think it contains? 
Results: 


one orange and little water and sugar 
25% orange and 75% carbonated water 
juice of one-half dozen oranges 

3 ounces of orange juice 
full-strength 

a quarter cup of orange juice 

none 

not much 

a small amount of orange juice 
one-fourth orange juice 

very little, if any, orange juice 
doubt it 

don't know 

not very much 

3 to 4 ounces of orange juice 

a pint 

most of it ao 
a little water mixed with orange Juice 
about a glass and a half 


aken from U.S. v. 88 cases (Bireley's Orange 


Py This eebmple ie! t 
fay. his exemple + GTiL. (1945) CLS. 0.4.0. 8alp. 


Beverage), Civil Action No. 


es: 


Better ways to word this question might have been: 


"This bottle holds sixteen ounces of a drink. How many ounces of ‘that 
would you say is orange juice?" 


"What percentage of this drink would you say is orange juice?" 


"What part of this drink - a quarter, a half, three-quarters, or what - 
would you say is orange juice?” 


Nevertheless, attempts to be clear and specific may sometimes go too far: 
Needless elaboration should be avoided. Judgement is needed as to whe- 
ther elaboration is excessive. Probably, nobody would deny that this 
example is excessive. 


Example 5.15 


How well do you get along with the women you married and regularly live 
with - that iss your wife? 


If, in the attempt to be specific, an example is given, it is desirable 
to give more than one and to make them as neutral as possible. The fol- 


lowing example of a question containing a single illustration will pro- 
bably not be very effective. 


Example 5.16 


Do you have respect for national political figures - such as 
Pierre. Trudeau? 


Respondents' opinions of Mr. Trudeau would tend to colour their answers. 
Rather than thinking of "national political figures" in a general sense, 
the tendency may be to think just of Mr. Trudeau. 


Often, it is necessary to use technical or legal terms. Whesethis oc- 
curs, definitions or explanations are recommended and care must be used 
to avoid any patronizing, condescending or official sounding tones. 
Consider the following question to be used in a survey of the general 
population in a medium-sized town. 


Example 5.17 


Are you aware of the impending amalgamation of surrounding constituen- 
cies into the 'New Metro' area? 


It is likely that many respondents will not understand terms like 'im- 
pending amalgamation’ and ‘constituencies’ and that rigorous explana- 
tions or alternate wording will be necessary. Explanations of terms 
such as these can be easily and tactfully presented. Difficulties 
generally arise from the elaboration about simpler or more common terms. 


The next example deals with a common process that directly affects 
most people. (Payne, page 116) 


et: 


Example 5.18 


How do you feel about your income tax, that is, the amount you have to pay 
the government on the money you take in? 
Since paying income tax is such a common experience, some may be offended 
by the inclusion of a definition. One way to avoid such a reaction is to 
give the explanation before the term itself. 


How do you feel about the money you have to pay to the government on the 
money you take in during the year, that is, your income tax? 


Providing the explanation of the term first forces the respondent to con- 
Sider the idea or concept in the fashion dictated by the researcher, and 
presents the definition as if it were a new discovery. The net effect 
seems to be an 'un-offended' respondent. 


A similar approach seems to hold for the ordering of ideas in a question. 
The respondent should be forced to consider conditions by placing the 
conditions at the beginning of the question. Placing the key idea first 
could lead respondents to consider only the idea and respond immediately 
without listening to or thinking about the remainder of the question. 

The following example presents two orderings of a single question. The 
first ordering could lead the respondent to answer immediately without 
considering the hypothetical constraint 'if you were forced to leave now’. 


Example 5.19 


In what area would you look for another house/apartment, if you were 
forced io Jeave, the one you, are in now? 


If you were forced to leave the house/apartment you are in now, in what 
area would you look for another? 


It is better to put the key idea last, with any explanations or conditions 
preceding it. 


As well as ensuring that the ideas in a question are clearly presented and 
defined, the researcher must ensure that the terms and or definitions will 
mean the same thing to all respondents. Without a common understanding of 
a concept, results cannot be aggregated. It is like adding together 


apples and oranges. 


Many surveys include a few questions on respondent characteristics. This 
information is usually requested to classify respondents into groups for 
The best approach for such questions is, of course, to use simple 


analysis. ; 
E There is, however, another consideration to be made and 
b] 


direct wording. Cy | 
that is the use of standard classifications and concepts. 


Why bother to standardize? In longitudinal studies ye et teeter 
tions and concepts make it easy for the respondent re eae e in fe 
tion quickly, since he follows the same eee Pa ser pre of 

time or short term studies, standardization al ib eae Te eee 
compared with other studies which used the same stan : 
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and again, it allows the respondent to produce the information easily. 
Finally, standardized definitions and concepts allow for wider use of 
the collected data in 'secondary' analysis. 


Statistics Canada has frequently been asked for advice on wording of 
questions on respondent characteristics. The Standards Division (Statis- 
tics Canada) has, therefore, produced a "Social Concepts Directory for 
Statistical Surveys". 


The directory suggests a number of standard wordings for asking age, 
family composition or makeup and size, on through to more complex topics 
such as education and occupation. A copy of the Directory can be obtain- 
ed by writing to the Federal Statistical Activities Secretariat, Statis- 
tics Canada, Ottawa, Ontario, K1A OT6, or the Standards Division, 
Statistics Canada, Ottawa. It can also be obtained through the local 
Regional Advisor of Statistics Canada in St. John's, Halifax, Montreal, 
Ottawa, Toronto, Winnipeg, Regina, Edmonton or Vancouver. 


5.5 Built-In Assumptions 


It is desirable to make sure that a question is applicable to all res- 
pondents who encounter it. Difficulties can be avoided by proper lead- 
in questions which include whether the respondent is qualified to answer 
it. For example, the question "How old is your wife?" should be prece- 
ded by questions that determine: (a) that the respondent is a male of 
marriageable age, and, (b) that he admits to having a wife. 


Questions should not assume that the respondent has knowledge or aware- 
ness in very specific areas. Such questions could embarrass or even 


annoy some respondents while others will claim knowledge they don't 
have, so as not to look foolish. 


Example 5.20 


Do you generally approve or disapprove of the policies of the military 
government of Peru? 


Do you think that incineration at 1600 degrees Celsius for 30 minutes 
is an adequate way of disposal of polychlorinated biphenyls? 


A common assumption among designers of business survey questions is that 
the respondent can readily give answers that may in fact require a con- 
siderable amount of work to complete. Questions on production, sales or 
amounts or values of raw materials, may refer to different time periods, 
different levels of detail, or different geographical boundaries, from the 
records kept by the company. Some prior discussion with officials of re- 
spondent companies might prevent this problem from arising. Alternatively, 
this is a problem that could be studied in a questionnaire pretest. Al- 
though it is best to adhere to a recognized set of standard concepts, “Lt 
may be necessary to accept reference-periods, areas, disaggregations, etc., 
that apply to the largest number of potential respondents, or to the 
largest or most important respondents, the ones the Survey designer least 
wants to irritate or most wants to get accurate answers from. 
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5.6 Double-Barrelled Questions 


In developing a questionnaire it 1s essential to avoid those questions 
which have two or more questions '‘nested' within them. These are gene- 
rally referred to as double-barrelled questions. Problems arise for 
both the respondent and the researcher when dealing with such questions. 
Respondents become confused in trying to understand and answer the ques- 
tion especially when they have different answers for each part. 


Researchers have difficulties interpreting the responses to such ques- 
tions since it is not clear to which sub-questions responses apply. The 


following is an example of a double-barrelled question with some pos- 
sible responses. 


Example 5.21 


Do you plan to leave your job and look for another one in the coming 
year? 


Possible responses: 


(i) Yes I plan to leave my job and no I don't plan to look for another; 
(ii) Yes I plan to leave my job and yes I plan to look for another; 
(iii) No I don't plan to leave my job and yes I plan to look for another; 
(iv) No I don't plan to leave my job and no I don't plan to look for 
another. 


If the researcher thought this question through and provided all of the 
possible responses as categories for the respondent to choose from, this 
double-barrelled question would not pose a problem for either the res- 
pondent or the analyst. However, if only two check boxes labelled ‘yes’ 

and 'no' were provided, and this is the usual case with such questions, 

how can the respondent who 'plans to leave his job and not look for 

another' indicate his response? If he checks both 'yes' and 'no', what 

is the researcher to infer? Is the intended response option (i) or 

option (iii)? 

One indicator of the likelihood of a 'double-barrelled' question is the 
appearance of the conjunction ‘and' or ‘or' in the question. The best way to 
avoid confusion is to replace single questions containing aug. 5. Gf, OF Ss With 


two or more simpler questions. 


5.7 Hypothetical Questions 


In general, it is better to avoid hypothetical questions, Suck as "what 

would you do if" or "would you like to do this” when asking respondents 
about their expected behaviour in particular Situations.— aoe to 
these kinds of questions have been shown to be poor predictors of actua 


behaviour. 


e United States National Bureau of Economic 

"Which of the following products do you 
When interviewed six months later, the 
had actually purchased ranged 


In a 1958 survey by th 
Research, respondents were asked, 
plan to buy within six months? 

proportion of those planning to buy who 
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from a low of 20.2% (for air conditioners) to a high of 57.7% Cer aueLo- 
mobiles). For most items, the proportions were in the 20%-40% range. 
Even when asked “which ... do you definitely plan to buy over the next 

12 months or so?", a follow-up interview 12 months later of those who had 
indicated a definite plan to buy yielded proportions ranging from 40.0% 
(for garbage disposal units) to 75.5% (for automobiles) with almost all 
other proportions, for 11 other items, in the 40%-60% range. (Source: 
Juster, F.T., "Anticipations and Purchases", NBER paper number 79, 

1964, Table 2, pages 22-26.) 


In general, it seems safer where possible to make predictions on tHe 
basis of projections from past behaviour. 


In some circumstances, however, hypothetical-situation questions may have 
some validity. For example, a hypothetical question may be appropriate 
if the situation is quite simple, or if the situation is very similar to 
one the respondent has been in at some time. Hypothetical questions can, 
with care, be used in third-party descriptions. For example, a descrip- 
tion is given of a situation with accompanying circumstances, happening 
to some one else and the respondent is asked for advice on what the other 
person should do. The kind of hypothetical questions to be avoided are 
the complex ones the respondent has likely never considered and would 
have no idea of all the details that should be considered to make infor- 
med choices. Answers are likely to be shallow and ill-thought-out,; and 
far from what the respondent really would do. Also, respondents may suffer 
from a degree of "social-desirability" bias. 


If there seems to be no alternative to asking a hypothetical question, 

it seems best to offer a wide choice of alternatives, rather than ask the 
open-ended "what would you do?" For example, "what would you do if you 
lost your job tomorrow?" might be better expressed with a list of several 
choices of courses of action. A further question along the lines of "is 


there anything you definitely would not do?" might help to make the pat- 
tern of choices clearer. 


5.8 Ease of Reading Questions in Personal or Telephone Interviews 


It is generaly preferable to give interviewers the complete questionnaire 
with the exact wording to be used. If the questions aren't complete, in- 
terviewers will improvise their own wording. Control over consistency of 
wording will be lost and this may lead to answers that cannot be compared. 
Interviewers should be encouraged to read questions exactly as worded, 
every time. Some of the following additions to the questionnaire may help 
make it easier for interviewers, especially from the point of view of 
approximating a conversational intonation. 


Spell out abbreviations unless they are very common ones all 
respondents are likely to understand, or ones that have al- 
ready been well defined earlier in the interview. This will 
help prevent the interviewer stumbling or pausing at having 
to make small decisions. "RCMP" would probably be accept- 
able, while "PCB" would probably mean something only to 
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chemical engineers. "BNA Act" would be understood by only a 
small proportion of the general public (and an even smaller 
Proportion could be expected to know much about cts 


Underscore critical words to ensure uniform emphasis by 
interviewers, and uniform interpretation by respondents. 


Example 5.22 
Now we'd like to talk about last week. Did youd (ame 
This will help emphasize a change in time reference. 


Be careful in using commas, colons and dashes. They might 
cause a break in the interviewer's verbal flow and may 
prematurely lead the respondent to assume that the question 
is finished and thus start to answer. 


Pretest the questionnaire to see if the wording is accepted 
and understood in a consistent manner by interviewers and 
respondents. 
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SEQUENCING 


6.1 General Concerns of Sequencing 


Once the researcher has translated his objectives into specific ques- 
tions, he must now be concerned with the ordering of these questions 

into a logical sequence, a concern which, if ignored, may have negative 
effects on the quality of data resulting from the survey. A logically 
structured questionnaire will be beneficial to the satisfaction of 

survey objectives by improving both the quality and quantity of response, 
not only to specific questions but also to the entire questionnaire it- 
self. Overall, question sequence must satisfy the respondent, the inter- 
viewer (when applicable), and the researcher. 


With respect to the respondent, the sequence should be designed initially 
to provide and later to maintain motivation to respond to the question- 
naire. Even for well motivated respondents, interest may diminish if the 
questionnaire skips from topic to topic irrationally, if many questions 
seem irrelevant, or if the interviewer gets lost in a complicated, un- 
clear questioning sequence. Questionnaire sequence should also overcome 
any suspicions or doubts that the respondent may have concerning the 
legitimaey of the survey. The use of basic principles of good sequencing 
may even facilitate the respondent's. recall of-specitigatens. oa. 10a1- 
tion, good sequence may aid in the completion of self-enumerated ques- 
tionnaires through the use of adequate and easy-to-follow instructions. 


For the interviewer's benefit, questions should flow in a clear and or- 
derly manner to ease in the administration of the questionnaire. Precise 
instructions should be included on how to move ahead, how and when to 
skip certain questions, and so on. 


Since the researcher's prime concern should be to obtain data of high 
quality, the ordering of questions should ‘not ‘lead ‘to, any. ibdas in) the 
data. For example, questions regarding the awareness of a concept should 
precede any other mention of that concept. The researcher also should 
try to avoid conditioning respondents in the early questionning to a 
frame of reference which could bias responses to later questions. In 
addition, non-response bias will likely be diminished if the order is 
such that specific questions are unlikely to be missed. 


6.2 Some Specific Aspects of Sequencing 


6.2.1 Instructions and Requests for Cooperation 


In most surveys, a potential respondent is under no compulsion to respond 
to a particular questionnaire. Even where response to a survey is obli- 


gatory, the legal requirement to respond is rarely enforced and may not 
even be known by the respondent. 
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Consequently, it is extremely important that the respondent's interest be 
aroused to motivate him to think carefully about or research the answers 
to the set of questions asked so that high quality data are better as- 
sured. Thus, the purpose of the Survey and a request for cooperation 
Should be included with the first information provided to a potential 
respondent, either orally by an interviewer (based on instructions con- 
tained in an interviewer's manual) or written in an introductory letter 


to the respondent included with the questionnaire or sent in advance of 
the questionnaire. 


Depending on the subject matter of the Survey as well as the respondent's 
own concerns, assurances of confidentiality of the information provided 
(if such assurances can in fact be made) should be contained in the open- 
ing remarks to the respondent. Confidentiality assurance is especially 
important when the survey contains questions that an individual perceives 
as personal or requests for information which a business establishment 
would not want a competitor to obtain. The researcher is indeed trying 
to "sell" the survey to the respondent, confidentiality being part of the 
Sa ese pat chi. 


For those surveys completed by the respondents themselves, instructions 
for return procedures and, where applicable a return address should be 
provided, not only in an introductory letter but also on the question- 
naire itself. The inclusion of the return address on the questionnaire 
1s recommended since introductory letters can easily be mislaid or 
separated from the main body of the questionnaire. The deadline by which 
respondents are to return completed questionnaires may also be included 
along with general instructions on how to complete a questionnaire and 
definitions of key concepts. 


Examples of questionnaires with their instructions and request for 
cooperation are shown in section 7. 


6.2.2 Identifying Information 


Some survey identification will be required on a questionnaire. Tne (tol= 
lowing identifiers all serve to establish specific information about a 
survey for reference purposes for the respondent or the interviewer: the 
title of the survey; the name of the survey sponsor; the name, address 
and/or telephone number of the interviewer or another representative of 
the sponsor to be contacted if there are problems or questions about com- 
pletion of the questionnaire. These items should be placed in a location 
separate from the survey questions themselves to avoid confusion. 


Each questionnaire in a particular survey should normally havi Doge e ee 
for its unique identification to facilitate survey ROE ee ne 
trol numbers may be needed for purposes of non-response contro af or fo 
low-up purposes) or for estimation procedures based on ae sample esign. 
Interviewer identification and place, date and time of the beers ey my 
be used for quality control procedures on interviewers ae ae a wage 
the interview has actually taken place, reinterview ae pet i 8 
check on the quality of an interviewer s work). The name, addre 
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telephone number of the respondent and/or the business establishment may 
serve to aid in obtaining further information, in clarifying information 
already obtained or in monitoring non-respondents for follow-up proce- 
dures. Colour coding of questionnaires may be used to easily distinguish 
questionnaires going to different segments of the sample, such as large 
businesses and small businesses or questionnaires completed in French and 
in English. 


Examples of survey identifiers from the 1976 Work Experience Survey 
managed by the Special Surveys Coordination Division, Statistics Canada 
and the Survey of Enrolment in School Board Adult Education Courses spon- 
sored by the Vocational and Continuing Education Section of the Educa- 
tion, Science and Culture Division, Statistics Canada follow. In these 
and most other cases, such identifiers are placed at the beginning of 

the questionnaire before the main subject matter of the survey. 


Example 6.1 


1976 Work Experience Survey (Special Surveys Coordination Division, 
Statistics Canada) 


WORK EXPERIENCE SURVEY 1976 


DOCKET NO. PAGE LINE P.S.U. GROUP CLUSTER 


Mo: (lo oo 


Example 6.2 


Enrolment in School Board Adult Education Courses (Vocational and 


Continuing Education Section, Education, Science and Culture Division, 
Statistics Canada) 


e ee Statistics Canada Statistique Canada 


Education, Science and Culture Division Please return 
Vocational — Continuing Education Section rob 1 
ENROLMENT IN SCHOOL BOARD ADULT EDUCATION COURSES 


JULY 1, 197__ to JUNE 30, 197__ 
Authority Statistics Act, Chapter 15, Statutes of Canada 1970-71-72 


Name of school board and oaiviiee 


Dace 


Addres 
ress Telephone 


Director of adult/continuing education i i 
Reporting officer 
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6.2.3 The Opening Questions 


Any initial motivation resulting from the introductory statements on the 
purpose of the survey must be sustained by the opening questions of the 
questionnaire. Thus, the first question must be sufficiently interesting 
to motivate the respondent not only to answer them but also to continue 
with the remaining questions. Above all, the questions should be related 
to the subject matter introduced in the purpose, described earlier to the 
respondent. Respondents may be under no obligation to answer the ques- 
tions posed and even those who are obligated may not provide accurate, 
well considered responses. Relevant, interesting questions at the outset 
may be the catalyst necessary to inspire the respondent and put him in a 
position to provide responses more willingly because of a real interest 
in the research problem and its effective implementation. On the other 
hand, seemingly irrelevant questions may cause the respondent to wonder 
whether the real purpose of the survey is to sell something. 


Opening questions should also be much like the opening problems on a high 
school examination: they should be sufficiently easy to give the respon- 
dent confidence to proceed further. Difficult or especially threatening 
questions in the early stages may be enough to cause non-response in the 
case of an initially apprehensive or unwilling respondent. Generally, 
the opening questions should put the respondent at ease. 


Thus, the 1976 Work Experience Survey began by asking questions about the 
respondent's work experience in the last year, concerning whether the 
work was full-time or part-time, the number of days per week and hours 
per day usually worked and so on. The 1975 Survey of Selected Leisure 
Activities, managed by the Special Surveys Coordination Division, Statis- 
tics Canada for the Secretary of State Department, started with questions 
on participation in eleven selected but fairly common leisure activities. 
The National Survey of Driving Habits, sponsored by the Road and Traffic 
Safety Branch of Transport Canada opened with questions to obtain infor- 
mation on the licence held by respondents, the number of miles driven in 
the previous twelve months and driver education courses taken. , In all 
these examples, the opening questions were related to the subject matter 
of the survey, were sufficiently easy to answer and generally interesting 


enough to most respondents. 


A technique used on occasion in surveys involving an interviewer is to 


begin in a conversational manner and thus build a rapport with the res- 


pondent by getting him immediatel 
hand. An example from Warwick an 
the use of an open-ended question. 


y and directly involved in the task at 
d Lininger (1975) illustrates this with 


Example 6.3: Warwick and Lininger, page 149 


What would you say are the main differences 


! di 
'Let's talk about schoo at they were like when you went 


between schools nowadays compared to wh 
to school?" 
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In some surveys, the opening questions may serve to establish that the 
respondent is a member of the survey population. For example, in Trans- 
port Canada's National Survey of Driving Habits, at the initial “comeact 
with a responsible adult in the household, a list was made of all those 
persons living permanently in the household who had a licence to drive a 
motor vehicle. After the random selection of a respondent, the opening 
question addressed to that respondent confirmed whether he had a valid 
licence to drive a motor vehicle. 


The opening questions also serve as a learning process, educating the 
respondent on his expected role concerning the topics raised, the detail 
required in answers and how to answer questions. This further emphasizes 
the earlier recommendation that the initial questions be easy to answer. 
The respondent is required not only to become attuned to the topic intro- 
duced on the questionnaire but also to learn what is required of him as a 
respondent. A suitable sequence of opening questions will attain these 
objectives. 


6.2.4 The Flow of the Items 


When considering the sequence of questions appearing in the main body of 
the questionnaire, the researcher should (design: the -questionnalresco 
seive the Logie of the respondent rather than applying his: owneleerc re 
the flow of questions. This will establish and maintain consistent 
frames of reference for the respondent and assist recall. What may seem 
logical to the survey designer who is very familiar with the subject mat- 
ter of the survey because of extensive background research, may not seem 
at all logical to the respondent on whom the questionnaire is suddenly 
thrust. Suitable question sequence will often allow respondents to anti- 
cipate their answers to the subsequent questions because such questions seem 
logical as a next step for discussion. A pretest of the questionnaire on 
typical respondents is recommended as the means for determining whether 
the flow of items is in fact perceived as reasonable by the respondents. 


Not only should the general flow of questions appear smooth to the res- 
pondent, but within the flow, the respondent should also be able to see 
the relationship of the questions to the stated objectives. Questions 
should relate to the overall purposes. Otherwise, suspicion may arise 
in the same way it could have with irrelevant opening questions. Ques- 
tions for which the only purpose in obtaining answers is that of being 


"nice to know", and which are only distantly related to the objectives, 
should be avoided. 


A technique often used to aid the smooth flow of question topics is that 
of brief transitional explanations or transitional questions. These 
methods help to introduce new topics as well as aid respondents in mov- 
ing easily from one frame of reference to another. Such transitional 
explanations or questions may be used to justify the inclusion of cer- 
tain questions which may appear at first to be irrelevant or too per- 


sonal. They thus serve to orient the respondent to the immediate 
objectives of a set of questions. 
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The 1976 Work Experience Survey serves as a good example of a question- 
naire with transitional explanations that help provide a smooth flow to 
the questions. Section A, consisting of a series of twenty-seven ques- 
tions, begins with "I'd like to begin by asking you some questions about 
your work experience in the past year," while Section B open with "Now 
I'd like to ask you some questions on your feelings about work". The 
respondent is reoriented to a new topic. Without the explanation, the 


change may appear to be too abrupt to the respondent, a situation which 
may lead to confusion and disorientation. 


Warwick and Lininger refer to "chronological lead-ins" which can be valu- 
able memory aids for questions on such topics as migration, job history 
or a respondent's perceptions of attitudes from his past. By focusing 
on events preceding those being pursued in the questionnaire, the respon- 
dent may be able to put himself into the proper time reference to remem- 
ber specific details more easily. For example, questions on job history 
could be preceded by a question on the date of completion of formal 
schooling or the date of a move to the respondent's current residence 

Or Caty. 


Lists of questions with exactly the same form in a long sequence should 
be avoided where possible or broken up by a different type of question to 
avoid boredom, irritation or putting the respondent into a particular 
response set where he doesn't really think carefully about the particular 
responses and just answers in the same manner as before. 


Ge... 5) The! Location of Sensitive Ttems 


Questions which are likely to be considered sensitive by respondents, be 
they concerning topics of a personal nature or subject areas that would 
be valuable to a firm's competitors if they could obtain the informa- 
tion, should be located in a section of the questionnaire where they are 
most meaningful in the context of other questions. In such a location, 
they may not appear to be out of place and are then somewhat desensi-_ 
tized. Income questions are usually perceived as being somewhat sensi- 
tive by respondents and thus generally generate higher than average non- 
response rates. However, if such questions are put within the context 
of economic conditions, job history or work experience, their relevance 
may seem more evident to respondents who may then be more willing to 
provide the information. In the 1976 Work Experience Survey, questions 
on job satisfaction which are potentially sensitive were placed niet mgor 
ately after more factual questions on work (type of work done, length o 


time on the job, and so on). 


Generally, sensitive questions should not be placed patentee a3 wes 
a questionnaire since at this time, the research is primari ‘| pices: 
with making a valiant effort to get the respondent intereste _— 
answering the questionnaire. At such a time, a sensitive ques ened 
could cause the respondent to give up completion of os SRE: oe ; 
An argument raised for putting sensitive questions op: atepic sagen 
questionnaire is that any hostility evoked by them wi n 
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further questions. Most of the information will already have been ob- 
tained. An argument against putting them right at the end of the ques- 
tionnaire (especially in long questionnaires) is that respondents may 
become tired or impatient and be more likely to refuse. The most appro- 
priate place is within the context of other questions. 


In a personal interview, sensitive items should be introduced at a point 
when the respondent will have developed some trust in the interviewer. 
The interviewer's own confidence in the study and her own capabilities 
will also aid in eliciting response from sensitive questions by putting 
the respondent at ease. 


In view of the above discussion, a prétest is the) best) vehielestorsde. 
termining first, whether any questions are perceived as being sensitive 
and secondly, the best location in the overall sequence for such 
questions. 


6.2.6 Classificatory Data 


Classificatory data describe the person/family/household/firm/plant/ 
institution by various economic, social or other characteristics that 
relate toithe subject of the research. Some examples) are: age wmsex, 
marital status, position in a household, education, occupation, income 
and number of employees. They are usually used for classification 
purposes in the tabulations or analysis. In somecases, one or more 
characteristics may be used in ratio estimation techniques which make 
adjustments to survey data according to (a) particular auxiliary vari- 
able(s) which is (are) correlated. to the variables of interest. For 
example, in the Labour Force Survey designed by the Labour Force Survey 
Division and the Household Surveys Development Division, Statistics 
Canada, age and sex are obtained for each respondent and then used with 


Census age-sex projections for a province to adjust survey data using 
ratio estimation. 


Classificatory data are usually requested in the latter part of a ques- 
tionnaire since they often do not directly and obviously relate to the 
survey objectives. In many cases, an explanation given to respondents 
as justification for their inclusion is that such questions are to be 
used for classification purposes only. These questions need not all be 
asked together and, in fact, it may be more reasonable to locate many in 
relevant sections of the questionnaire. For example, income questions 
could appear with questions on work experience or employment status. 


Generally, only questions that are expected to be used for specific anal- 
ses should be asked. For example, a reason often given for a question on 
marital status is that "It's always asked". This is insufficient reason, 
especially since the concept of response burden is becoming more and more 
of a concern to survey designers and respondents. 


The Social Concepts Directory for Statistical Surveys prepared by the 
Standards Division provides questions and suggested answer categories for 


many of the items frequently included in surveys for classification 
purposes. 
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6.3 Specific Strategies 


6.304 'Filters 


A filter question is used to exclude a respondent from a sub 
tion sequence if those questions do not apply to him because 
particular characteristics or circumstances. 
Situation. 


sequent ques- 
of his own 
Example 6.4 shows such a 


Example 6.4 
(1) Are you now married? O Yes 


O No > Skip to question 8 


Please answer the following questions about you and your spouse. 
(follow with questions 2 to 7) 


The skip instruction serves to keep those unmarried respondents answering 
only those questions which apply to them. They do not have to read and 
consider as irrelevant, questions that do not apply. -This*helps* maintain 
interest as well as avoid the situation where the respondent may develop 
the impression that the questionnaire must have been sent to him by mis- 
take, and as a result fails to complete the remaining questions. 


At times, one will see the use of a conditional question to make a ques- 
tion relevant to all respondents. For example, "If you are working or 
looking for work, what is your main reason?" would be applicable to all 
respondents. Those who are neither working nor looking for work would 
ignore the question while others would answer it. The main disadvantage 
with the use of a conditional question is that a blank answer can be 
duterpreted either as a non-response to thé question or asavchapacter— 
istic of a respondent who neither works nor is looking for work. In the 
latter case, the question is not applicable to these people and is legi- 
timately left blank. It is thus impossible to distinguish between a 
non-response to the question and a "not applicable" blank answer. It 
would be dangerous and possibly highly misleading to infer that all 
blanks were a result of people to whom the question did not apply. 
The use of a filter question is recommended over a condition being built 
into the question by means of a statement beginning with vif 2's In 
this case, the filter question could be, "Are you now working or looking 
for work?" Respondents answering "Yes" could be directed to answer the 
question that obtains the main reason for working or looking for work. 
If the response is "No", the respondent could be instructed to skip to 
the next appropriate question. Another ayes: although not as poe ul 
factory, is to include an answer category ("not working or looking or 
work" in the example given above) which contains the alternative 


condition. 


Generally, filter questions relying on easy-to-follow skip instructions 


é inuati ints 
direct respondents and interviewers to appropriate continuation pol 
and serve as an effective sequencing technique. 
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Gy Sas. sl lheekinnel Strategy, 


Two of the major criticisms of public opinion or attitudinal research are 
that uninformed persons or persons who are informed but do not carefully 
think about the issues raised in a question or set of questions may not 
be distinguishable from those who carefully consider the issue and that 
often the reasons for opinions are ignored. George Gallup, as Director 
of the American Institute of Public Opinion, proposed a question sequence 
which tried to alleviate these criticisms (Gallup, 1947). Because the 
technique probed five different aspects of the respondent's thinking, it 
has been called the "“quintamensional" approach or plan. 


Basically, the quintamensional design proceeds from questions that are 
broad and unstructured to those that are more specific by using five 
categories of questions. Depending on the topic and circumstances, not 
all five categories may be needed while it is also possible that many 
questions could be asked within each category. Thus, the approach is 
flexible depending on the given situation. 


The five categories are as follows: general awareness of the topic, 
uninfluenced general attitudes on the subject, specific attitudes, 
reasoning behind the attitudes, and intensity of feeling. 


The opening category of questions serves as a filter to find out whether 
the respondent is aware of the topic. A single question requiring a 
"Yes" or "No" answer may be used here, although a "Yes" answer does not 
provide proof that the respondent is actually aware of the subject since 
some respondents may be reluctant to confess ignorance. Usually, further 
probing is required to determine this, probing which strives to determine 
just how much information the respondent does have, if any, concerning 
the issue. Open response questions are frequently used to determine the 
general awareness. 


Open response questions are often employed in the second category to 
explore in a somewhat unstructured fashion, opinions of the respondent 
himself. Such questions attempt to explore the direction of his 
thinking. 


The third category, approaching the topic more specifically, includes 
questions on particular proposals or opinions and employs dichotomous 
or multiple choice items. The researcher, by using the first two cate- 
gories as a frame of reference, is in a better position to understand 
the specific opinions put forth by respondents in this third category. 


In the fourth category, the questionnaire designer is trying to learn 
the reasons behind the views expressed earlier. Open-ended questions 
should provide this information. 


The last category determines the intensity with which the opinions al- 
ready expressed are held and is the most specific set of questions 
posed in the quintamensional design. 


— §64°— 


Example 6.5 (Kahn and Cannell, 


1957) illustrates the 
question for each category. approach using one 


Example 6.5: (from Kahn and Cannell, page 159) 


(1) How do you feel this country is getting along in its relations with 
other countries? 


(2) How do you think we are doing in our relations with Russia? 


(3) Do you think we ought to be dealing with Russia differently from the 
way we are now? 


(4) (If Yes) What should we be doing differently? 


(5) Some people Say we should be tougher with Russia, and others think 
we are too tough as it is. How do you feel about it? 


The quintamensional sequence of questions trie to ascertain the respon- 
dent's frame of reference before obtaining information on his opinion. 
Because it allows the respondent some freedom of expression, he need not 
feel cheated at not really having had a chance to have his say on an 
issue. The respondent is free to verbalize all salient aspects of the 
issue. The sequence is effective in focusing on the finer details of 
the respondent's perceptions through the combined use of open and closed 
response types, allowing freedom of expression while still enabling the 
researcher to obtain specific data which can be compared across respon- 
dents. Depending on the situation, the quintamensional design may even 
aid the respondent in recall because of its gradual focus on the area of 
concern. 


6.3.3 The Inverted Funnel Strategy 


Kahn and Cannell discuss situations where an inverted funnel sequence may 
be appropriate. Basically, this approach commences with specific ques-~ 
tions and proceeds to more general questions on the respondent's level of 
knowledge about the issue. The respondent is forced to think through his 
opinion in a number of areas which make up the issue. 


The technique is appropriate when the respondent 1s likely to have thought 
little or not at all on the topic or have very little information about 
it. In such a case, the researcher is trying to ensure that the respon- 
dent has considered certain points before answering the broad question(s) 
on the topic itself or that all respondents have an equal frame of refer- 
imensions before reaching the general issue. An exam- 
nell discusses an evaluation of a work Situation 
quence takes the respondent through very specific 
the physical conditions of work, the 
Then, the general question posed is 
pany as a place to 


ence on specific d 
ple from Kahn and Can 
where the question se 
questions on the worker's foreman, 
content of the job and so forth. 

concerned with how the respondent feels about the com 


work, taking all the specific details into consideration. 
4 
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The inverted funnel strategy is also appropriate when the respondent may 
have such strong feelings about a general issue that any responses which 
followed such a question would be biased by them. Warwick and Lininger 
illustrate this with an example where the questions get information first 
on overall attitudes to a minority group, then on specific members of the 
group. The respondent's desire to remain consistent with his original 
judgement could affect later response. The inverted funnel technique 
attempts to alleviate this problem by posing the specific questions 
LPEsSts 


6.4 References 


Ferber, R.; Blankertz, D., and’ Hollander, S. Jr. ° Marketing Research. 
New York: Ronald Press, 1964. 


Festinger, L. and Katz, D., Eds. Research Methods in the Behavioral 
Sciences. New York: Holt, Rinehart and Winston, 1965. 


Gallup, G. "The Quintamensional Plan of Question Design". The Public 
Opinion Quarterly, Fall, 1947. 


Jahoda, M., Deutsch, M., ‘and Cook, S. Research Methods"in'Social Rela= 
tions Part Two: Selected Techniques. New York: The Dryden 
Press ’.°1951" 


Kahn, R.L. and Cannell, C.F. The Dynamics of Interviewing. New York: 
Wiley, 1957: 


Luck, D.J., Wales, H-G. and Taylor, D.A.. Marketing Research, 4th edition. 
Englewood Cliffs: Prentice Hall, 1974. 


Moser, C.A. and Kalton, G.J. Survey Methods in Social Investigation, 
2nd edition. New York: Basic Books, 1972. 


Oppenheim, A.N. Questionnaire Design and Attitude Measurement. London: 
Heinemann, 1966. 


Payne, S.L. The Art of Asking Questions. Princeton: Princeton Univer- 
sity Press 1951. 


Warwick, D.P. and Lininger, C.A. The Sample Survey: Theory and Practice. 
New York: McGraw Hill, 1975. 


= 8D 


LAYOUT 


Layout consists of arranging the contents of a questionnaire into an ef- 
fective and attractive package. Although this section applies to all 
questionnaires whatever the method of data collection, it applies parti- 


cularly to those questionnaires which are to be completed directly by 
respondents. 


There is no one best layout, as the way in which the questionnaire will 
be used, as well as other factors, will have an influence on the design. 
For instance, those questionnaires which are to be completed by inter- 
viewers need not contain as complete instructions as those which are 
completed by respondents. They can also contain questions which are 
used by the interviewer to screen suitable respondents from the general 


public, or questions which are completed by the interviewer from obser- 
vation only. 


Questionnaires which are to be completed directly by respondents should 
be more complete in every respect. There is no interviewer available 

to explain the purpose of the survey or clarify any of the questions. 
When the prospective respondent receives the questionnaire package, it 
must convey the entire message and contain the answers to most reasonable 
question regarding the identity of the sponsor, the purpose of the survey, 
the confidentiality of responses given by respondents, the urgency, com- 
pletion instructions, and the address to which completed questionnaires 
should be sent. 


7.1 Layout Considerations 


In planning layout, the following points should be considered: 


7.1.1 Appearance 


The effect which the appearance of the questionnaire has on a respondent 
will affect the respondent's attitude toward responding. The initial 
impression can influence the respondent to peruse the questionnaire. 
Then, once the respondent's interest has been aroused, it must be 


retained. 


7.1.2 Ease of Entering Data 


The layout should provide the easiest possible method for entering the 
data requested. As responding is a courtesy in most cases, it is only 
reasonable to make respondent burden as light as possible. 


7.1.3 Ease of Using Data 


sing the data collected. It is both time- 


Llatate su : 
The layout should facilitate cessing staf£.to have to search for ir- 


consuming and exhausting for p 
regularly-located entry spaces. 
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7.1.4 Reducing Tendency to Error 


The layout should help reduce any tendency to error in entering or using 
the data. Respondents should not have to guess in which space a parti- 
cular answer should be entered, which unit of measure to use, or whether 
only one or several response answers are expected to a multiple-choice 
question, etc. Obvious errors may be detected by processing staff but 
making corrections adds to processing costs. When answers in irregular- 
ly-located entry spaces are keyed, there may be a tendency for some 
entries to be missed. 


uted Size and’ Construction 


The physical size and construction of the questionnaire should lend them- 
selves to the conditions under which the data are entered. Question- 
naires which are to be completed in an office may have large pages with- 
out being inconvenient, while diary-type questionnaires should usually 

be packaged as a small booklet for the convenience of respondents. 
Questionnaires which are to be completed by interviewers may be designed 
for use with a clipboard. 


22 Desiten Principies 


There are a number of design principles which will assist in laying out 
a questionnaire: 


Tie “Simplicity and larity 


The first impression conveyed to respondents by a questionnaire should 

be simplicity and clarity. Apparent complexity (or a cluttered appear- 
ance) will tend to emphasize the effort required to complete the ques- 
tionnaire and may cause the respondent to delay completion, either tempo- 


rarily or indefinitely. The best approach is to be businesslike and 
polite. 


Normally, the front page should contain: 


- Identification of -the sponsor; 


- A message identifying the purpose of the survey, possibly in the form 
of a short letter from a senior person in the sponsoring organization, 

- General completion instructions, 

- Confidentiality to be given to the data provided, 

- Required date for completion, 

Instructions for return of complete questionnaire. 


If space permits and the page will not appear crowded, the opening ques- 
tions may also appear on this page. 
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The numbering of all questions serves several purposes. It assists the 
respondent in identifying where each new question starts and it assists 
the designer in controlling skip patterns where all questions may not 

apply to all respondents. The use of oversize numbers is often useful. 


The use of solid lines (known as "rules") between questions is not al- 
Ways necessary, and may not be desirable in some layouts. The use of 
non-essential lines should be avoided because they can clutter up the 
questionnaire and can be confusing to some respondents. Conversely, 
lines may be useful in separating questions which are closely packed on 


a page. In general, borders should also be narrow because they are 
wasteful of useable space. 


Some questionnaire designers use a pictorial device such as a line draw- 
ing or photograph which symbolizes the subject matter, prominently dis- 
played on the first page. Such a device will likely attract a respon- 
dent's attention and can be prepared in such a way as to represent very 
Leove cost 


ieee? instructions 


All instructions which concern the completion of questions should be 
placed directly above the question(s) concerned, or at the front of 
the questionnaire. 


When an instruction applies to a series of questions, some device should 
be used to signal when the instruction no longer applies. This device 
could be a bold horizontal line, another major heading, another instruc- 
tien. Sere. 


Instructions and introductory statements should be in a type style which 
is distinctly different from that used in the questions, so that they 
stand apart from the questions. 


Unless respondents are expected to answer all questions in sequence, it 
is useful to use arrows and/or directions to indicate alternate paths 
or skips in the answer pattern for different categories of respondents. 
These arrows and/or instructions should be in boldface type or a con- 


trasting colour so that they stand out. 


jac; 2) Simp le: Language 


It should usually be assumed that respondents ih laa educated or that 
they are pre-occupied or indifferent when completing a pape epee 
Respondents often live and work in an environment that is quite di teed 
from that of the researcher and may also have a different pattern o gic, 
as well as having different value systems and priorities. 


ould avoid technical terms, abbreviations or acronyms 
he respondents. A glossary of 
be useful if the use of 


Questionnaires sh 1i ¢ : 
unless these are known to be familiar to 
terms in the front of the questionnaire may 
technical terms cannot be avoided. 


ee 


7.2.4 Consistency of Style and Terminology 


Insofar as possible, the style of layout and wording should be consis- 
tent. Changes of style of layout which are used simply for variety may 
be interpreted to have a significance that was not intended. 


The same applies to the nouns used. Where several nouns can be used 
interchangeably, one of these should be selected and used throughout 
the questionnaire. Respondents may infer that a different meaning is 
intended where a synonym is used occasionally. 


The association between each question and its answer space should be 
unmistakable, so that entries are placed in the correct space. 


The layout of multiple-choice questions should be consistent within each 
group of questions. Where a change in pattern of layout occurs between 
groups of questions, this should be identified by a conspicuous, anstrucs 
tion; otherwise, respondents may assume that the same pattern continues 
throughout all multiple-choice questions. 


7.2.5 Logical Groupings and Sub-Titles 


Most questionnaires deal with several different topics, and may also an=- 
clude questions on both facts and opinions. Generally, questions should 
be grouped in clusters and arranged in a sequence which is logical to 
the respondent... Jf the subject matter arrangement appears. ito be bles 
cal’, -the respondent may become, confused or irritated. 


Each sub-division should be identified by a suitable short title,eand 
possibly a sentence or two to introduce the new topic area and explain 
which aspects, will be covered by the questions..to, follow. this darects 
the respondent's thinking in the desired direction. 


Where questions in different topics areas deal with a previously men- 
tioned subject, the second (and subsequent) questions on the subject 
should acknowledge this situation and clearly identify that the current 
question deals with a different aspect of the subject. This avoids the 
feeling that there is duplication of questions. 


7.2.6 Adequate Spaces for Answers 


In many cases, the amount of space provided for write-in answers will 
influence the length of the answer. 


Where the length of a normal answer is known in advance, the designer 

can ensure that sufficient answer space is provided. Inadequate space 

may cause the answer to be abbreviated or incomplete. This problem is 

more prevalent where the questionnaire is prepared by an operator who is 
not given adequate layout instructions. This results in short questions 
having excess space for answers and long questions not having enough space. 
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7.2.7 Suitability for Processing of Data 


Where the answers on a questionnaire are to be coded and captured on 


cards or tape, a computer systems analyst should be consulted early in 
the design phase. 


Some questionnaires lend themselves easily to pre-coded entry spaces, 
while others may require a coding column down the right edge of each 
page, separate work sheets or some other arrangement. 


Coding arrangements should not be conspicuous as they may confuse some 
respondents. 


Questionnaires using mark-sense entries or optical character recognition 
should be field-tested with the particular survey population to ensure 
that respondents know how to complete them properly. 


7222.8 Colones: ote Paper ands) nk 


Effective use can be made of coloured paper for the printing of ques- 
tionnaires, particularly in a light tint or pastel colour. Vivid or 
dark colours of paper should be avoided because of the reduced contrast 
between the paper colour and the printing which may render the printing 
very difficult to read under certain types of lighting. Different 
colours of paper can be used to identify different variations or sec- 
Lions Of a questionnaire: 


Small quantities of questionnaires are almost always printed in black 

ink because of the time and cost involved in changing ink colours on 

the press. For large press runs, the use of coloured ink becomes eco- 
nomically justifiable and can achieve a pleasant appearance and excellent 
contrast with the particular colour of paper used. Two contrasting 
colours of ink can be used effectively, but the cost becomes excessive 


for small quantities of questionnaires. 


7.2.9 Bilingual Layouts 


Where a questionnaire is to be produced in both languages, back-to-back 
or "tumble" layout or some similar separation makes the questionnaire 
easier to understand by all respondents. Alternate lines of English 


and French should be avoided in most cases. 


Where the physical segregation of English and French cannot be achieved, 
different type styles should be used for each. Instructions should then 
be enclosed in boxes to avoid the use of too many type styles on the 


same page. 


and French in side-by-side columns, the French 


“ lish y 9 o 
halle Serna rie t first because it usually requires 25-30% more 


text should be laid ou 
space. 
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Example 7.1 


The Canada Health Survey used a combination of physical tests and a 
self-completed questionnaire, two pages of which are shown on pages 71 
and 73. The questionnaire was dropped off and picked up by an 
interviewer. 


1. The sponsors are clearly identified on the cover of the 
questionnaire. 


2. The prominent title clearly identifies the subject matter. 


3. The introductory paragraphs reinforce the verbal explanation 
which was given earlier by the interviewer. The purpose of the 
Survey is described, cooperation is sought, and the means of 
ensuring confidentiality is explained. 


4, The series of photographs are eye-catching and they also reinforce 
the title. 


5. The date is specified on which the interviewer will pick up 
the completed questionnaire. 


6. The Docket No. and Person No. are included on the cover for 
administrative purposes during collection and data capture. 


Example 7.1]. 


The interviewer will pick up this questionnaire on 


8-3200-3 1.1-3-78 


lifestyle 
and 
your health 


Lifestyle has a direct influence on health. The effect may not be immediate, but everybody 


knows that risks taken today are paid for tomorrow, just as positive action results in better 
health 


The Canada Health Survey is collecting information on the state of health of Canadians At 
the interview we collect information on people's health now and in the past but, to plan 
effectively, we must also have indications of health in the future. That is why we ask that 
you fill in this questionnaire 


Your lifestyle is a private matter. To ensure your privacy, we have provided you with an 
envelope for your questionnaire All the information collected will be used in combination 


with answers from the other participants. No individual will ever be identified from the 
results 


Please take a few minutes to help us pian for the future needs of all Canadians 


a3 


Example Jigs 


Each page (or set of two pages) of the Canada Health Survey question- 
naire is devoted to one particular aspect of lifestyle and/or health. 
Page 6 of the questionnaire (on page 73) deals with tobacco. 


1. The subject matter is clearly identified, followed by a short 
sentence which establishes the desired frame of reference on 
the part of the respondent. 


2. The questions are clearly numbered and easy to follow. 


3. The entry spaces provided for numeric entries are distinctly 
different from those provided for tick marks. 


4, “Thesalternative answer paths. (Part “A or Part "'B") are clearly 
identified by the arrows in question 2. 


5. The skip instruction in question B3 instructs the respondent 
to ignore page 7 of the questionnaire which deals with additional 
details about smoking habits. 


6. The bold arrows at the bottom of the two panels clearly indicate 
that only one of these panels is to be completed by any one 
respondent. 


Example 7.2 


TO BACCO These questions are about your experience with tobacco and smoking 


1. Check any of the following which you now smoke daily. 


«(| pipes 
w{_ | cigars 
«| _] cigarillos 


OR » «o{_] none of these 


2. Do you smoke cigarettes daily? 


60s [_]: Yes — Answer the questions ws[ J. No — Answer the 
_— in PART “A” only - questions in 
PART “B” only 


PART “B” 


. What experience with cigarettes have 
you had? (Check one only.) 


PART “A” 


. Atwhat age did you start smoking cigarettes daily? ea[_ | Never smoked 


[_] Now smoke Go to page 8 
occasionally (Alcohol) 


3 Used to smoke 
occasionally 


606 
At age | 


. About how many cigarettes do you now 
smoke each day? 


Fis 
About a day 


. How far do you usually draw in the smoke? 


‘a? Used to smoke daily 


. At what age did you start smoking daily? 


| 
At age eel te | 


. At what age did you stop smoking daily? 


w[ _]} Only into my mouth 


[] Into my throat 


s Partly into my chest 


Deeply into my chest 


At age ss bof 


. About how many cigarettes did you 
usually smoke daily? 


About LL a day 


7. How far did you usually draw in the smoke? 


I'm not sure 


Has your smoking changed over the 
last 12 month? 


oo] Smoking more now 


[_]: Smoking less now 


617 » Only into my mouth 
vol ds Switched to a stronger brand a Only y 


2 Into my throat 
[_]: Switched to a milder brand LJ CY, 


: hest 
OR & «:[__] No change over the last 12 months L_]: Partly into myc 


ris Deeply into my chest 


[_]s I'm not sure 


Have you tried stopping during the last 12 months? 


8. Please check the one brand of 
cigarettes which you used to 
smoke in the list on the next page 


ee[_ |: Yes [_]. No 


Please check the one brand of 
cigarettes which you usually smoke 
in the list on the next page. 


Example 7.3 


The Labour Market Comparison Study was a supplementary survey to the 
monthly Labour Force Survey in March 1979. It was conducted by the 
Labour Force interviewer in a personal interview. 


1. The confidentiality of the completed questionnaire is stated. 


2. The control information is needed for administrative purposes 
during collection and data capture. These identifiers are dropped 
once the computer records have been "cleaned". 


3. Instructions to the interviewer which are unique to this survey, 
are clearly set out where the interviewer will see them before 
completing the questions. 


4, The paragraph in italics in question 10 is to be read to each 
respondent who falls into the specified sample. 


5. The entry spaces for numeric entries are a different shape from 
those for tick marks. 


6. The skip patterns in questions 11 to 16 are too complex for use in 
a self-completed questionnaire but are acceptable for use by 
trained interviewers. 


Example 7.3 


a+ Statistics Canada Statistique Canada 


LABOUR MARKET COMPARISON STUDY 


1 FORM NO. O06 
Survey date 3(0,317,9] Assignment vo. 4{ | 11] | 
Mo. re 


docket No. 21 | TT TT | 


HRD page- 
line no. 


5L1 16 


Given name 


1 = 


NTIAL j 
when com) leted | 


Surname 


TLL ae ei] 


INTERVIEWER 
Step A: 


INSTRUCTIONS 


Transcribe identification from the 
FORM 03 for the first (next) HRD page- 
line no. 15 years of age or over. 

(Do not include HRD page-line nos. 

with Code 2 in Item 40 of the FORM 03.) 


INTERVIEWER 
Step B: 


INSTRUCTIONS 


Complete the FORM 03-FORM 05 Sharing 
Agreement below with each information 
source for the FORM 03 and FORM(S) 05. 
(Do not complete the FORM 03-FORM 05 
Sharing Agreement on subsequent FORMS 
06 for this household. ) 


10 SHARING AGREEMENT 

"As I have mentioned, Statistics Canada and the 
Economic Council of Canada have designed this sur- 
vey because they feek that the information which 
you have provided will be valuable in determining 
ways to reduce the unemployment problem in Canada. 
Statistics Canada would Like your approval to share 
this Anformation with the Economic Council of Canada 
although you have the right to deny this sharing. 
May I have your approval tc share this information?" 


(Enter HRD page-line no. for each information 
source for the FORM 03-FORMS 05.) 


"Would you please sign here to indicate that 1 have 
necorded your response correctly." 


HRD page- 
line no. 


[I | 


HRD page- 
line no. 


[1] 


HRD page- 
line no. 


28 


HRD page- 


line no. 
oO 


Be No °C) 


"Although your answer is sufficient you may also write 
during the next month, to the Chief Statistician, 
Statistics Canada, Ottawa, to indicate that you agree 
on do not agree to share your information with the 
Economic Councik of Canada. Please include the 
following number in any connespondence ... 


'O 
@ 


Respondent's signature 


@ 
ue 


Respondent's signature 


5 @) 
©) 


Respondent's signature 


Respondent's signature 


(Read this respondent's Docket no. and HRD page- 
line no.) 


INTERVIEWER 
Step G: 


INSTRUCTIONS 
Complete Items 11 to 16 referring to 
the FORM 05 and to the RESPONDENT 
SELECTION TABLE when required. 


If a selected household member is 
NOT available for interview, arranae 


for a call-back. 


11 INTERVIEWER CHECK ITEM: ON FORM 05 


. If "Yes" in Item 10 
(worked last week)....... 10) go to SELECTION TABLE 


. If selected 
go to 45 


. If not selected...3C) END 


. If "No" in Item 10 ‘5 
(did not work).....es.00. O go to 12 


. If "Perm. unable to 
work" in Item 10 


12 INTERVIEWER CHECK ITEM: 


a ueaded 5) 25, 35 kas 
7 or 0 in Item 33 


. If selected 
go to 45 


. If not selected...3() END 


. If Codes 5, 6 or 
blank in Item 33 


13 INTERVIEWER CHECK ITEM: 
. If blank in Item 57 


. If ... has done nothing 
to find work in the past 
four weeke in Item CS et go to 14 


. If ... has done something 


to find work in the past 
four weeks in Item By mast) go to 17 


14 op... 


Yes 1c) go to 15 


WANT A JOB LAST WEEK? 


No 20) go to 16 


15 wAS THERE ANY REASON ... COULD NOT TAKE A JOB LAST WEEK? 


al Enter code and go to 24 


16 INTERVIEWER CHECK ITEM: ON FORM 05 
. If "Full-time" in 


. If selected 
go to 54 


. If not selected...3() END 


. If Part-time” or blank 
in Item 81 


“Puthority — Statistics Act. Chapter 15 
Bratutes of Coneds 1970-7) 77 


8-3400-43:05-02-79 
8-619-ECC-01 


Example 7.4 


The Survey of Inpatient Facilities for the Treatment of Emotionally 
Disturbed Children was conducted by mail. It was addressed to the 
administrators of the respective institutions. 


The use of a bilingual form eliminates the need to ascertain 
which language is used in each institution, but the layout 
appears somewhat cluttered as a result. 


The columnar arrangement of entry spaces along the right margin 
facilitates data capture and also is convenient for the respondent. 


The careful use of large brackets within the questionnaire 
provides easily-understood breakdowns of the various categories 
with a minimum of textual explanation. 


In general, the questions are more direct and business-like 
than on a personal questionnaire intended for completion by an 
individual. 


Example 7.4 


SURVEY OF INPATIENT FACILITIES FOR THE TREATMENT OF EMOTIONALLY DISTURBED CHILDREN 
ENQUETE SUR LES ETABLISSEMENTS DE TRAITEMENT DES ENFANTS AFFECTIVEMENT PERTURBES 
CANADA, 1974 


Is the name and address label! of your facility correct? Yes f7] No 5] 
Cette étiquette est-elle compléte et exacte? 


If "NO", please provide correct information below 
Si "NON", inscrivez les renseignements exacts ci-dessous 


NAME — NOM 
ADDRESS — ADRESSE 
POSTAL CODE POSTAL 
al ie [eal 


* PLEASE SEE DEFINITIONS — * CONSULTEZ LES DEFINITIONS 


| Historical and Administrative Data — Données chronologiques et administratives 


1. Please indicate type of your facility (check one only) 
Indiquez le genre d’établissement (ne cochez qu‘une case} + RESIDENTIAL TREATMENT CENTRE 
CENTRE RESIDENT/EL DE TRAITEMENT 
ALTERNATE LIFESTYLE RESIDENCE 
INTERNAT DE READAPTATION A UNE VIE NORMALE 


GENERAL HOSPITAL 
HOPITAL GENERAL 


nN 


UNIT JNA 
UNITE D‘'UN 


» 


HOOOoco: 


CHILDREN’S HOSPITAL 
HOPITAL PEDIATRIQUE 


MENTAL RETARDATION CENTRE INCLUDING HOSPITAL TRAINING SCHOOLS 
CENTRE POUR ARRIERES MENTAUX (COMP. ECOLE DE FORMATION) 


PSYCHIATRIC HOSPITAL 
HOPITAL PSYCHIATRIQUE 


a 


a 


OTHER (specify) 
AUTRE (précisez) 


A 


2. When did your facility commence operations for the treatment of emotionally disturbed childreny 
Quand votre établissement a-t-i/ commencé 4 s‘occuper du traitement des enfants affectivement perturbés? 


YEAR aoe 
ANNEE 4 te 
. 005 
3. (a) Are you licensed to operate a facility for the treatment of emotionally disturbed children? Yes 7) No pad 
Avez-vous un permis vous autorisant & exploiter un établissement de traitement des enfants affectivement i ee Non __! 
perturbés? oe 
(b) If ““YES"', is your facility licensed by a municipal or a provincial government’ (Check one only and name MUNICIPAL! CI 
the department or ministry that was directly responsible for the licensing) : MUNICIPAL 
Si OUI’, indiquez le niveau de gouvernement (ne cochez qu'une case) et donnez /e nom du ministére ou PSOWINEIALL 2 
service qui vous a délivré ce permis. PROVINCIAL 4 


NAME — NOM : 7 
is 7 At ee 


i 00s 
4. Please indicate type of ownership (check one only) 
hez qu'une case) INDIVIDUAL OWNERSHIP iT ] 
Indiquez /e genre de propriété (ne coc g perc hehe ak Re gall 2, 
. Proprietary PARTNERSHIP 2 
Ownership: (i.e. person, group of persons, agency or Privée a but lucratif SOCIETE EN NOM COLLECTIF | 
corporate body who isthe registered owner according CORPORATION J 
to the deed. CORPORATION Ll 
LAY (VOLUNTARY LAY BooY 4 
Propriété: (c.d.d. personne, groupe de personnes, SORE LAIQUE (BENEVOLE) [| 
organisme ou corporation qui, selon /'acte d’enregis- Voluntary RELIGIOUS : 7 
trement, est /e propriétaire en titre). A but non-lucratif ORGANISME RELIGIEUX | 
CORPORATION 6 
CORPORATION 
{ MUNICIPAL | 
Government d MUNICIPAL 
Gouvernement PROVINCIAL ‘ 


a PROVINCIAL 


w 
OTHER (specify) : je 


AUTRE (précisez) - « = = s 


5. (a) Is this facility @ part or a member of a chain of two or more facilities’ 


if fi ? 
Cet établissement fait-// partie ou est-1/ membre d'une chaine de deux établissements ou plus 


i He Fae 
(b) If ‘‘YES’’, please name the parent or controlling body 
Si OUI’, nommez |’établissement principal ou la corporation dirigeante 


NAME NOM 


— TPOSTAL CODE POSTAL 
ADDRESS ADRESSE ll, mre 


4—2301-57: 28-4-75 


2 Gee 


Examples 7.5 and 7 «6 


The Travel Survey 1977 was conducted by personal drop-off and pick-up 
by a Labour Force interviewer. 


i. The sponsor is clearly identified: 
2. The title is displayed prominently. 


3. A letter signed by the Chief Statistician explains the purpose 
of the survey, asks for cooperation and assures the respondent 
of confidentiality of responses given. 


4, The control information is needed for administrative purposes 
during collection and "cleaning" of computer tapes. 


5. Completion instructions are given at the bottom of the front 
cover. These instructions can be discussed with a member of 
each household when dropping off the questionnaire, and any 
queries answered immediately by the interviewer. 


6. The questions on pages 80 and 81 may appear complex at first 
olance Otis. 


(a) all questions are arranged in a single column and are clearly 
numbered; 


(b) each question is answered across the two pages for as many 
trips as apply. 


7. Entry spaces provided for numeric entries are a different shape 
than those provided for tick marks. 


Example 7.5 


iv Statistics Canada Statistique Canada 
TRAVEL SURVEY 1977 


Dear Respondent, 


The purpose of this survey is to provide information on the travel and transportation 
patterns of Canadians. tt is being conducted for Transport Canada, who will be using the 
data to better understand passenger transportation needs and flows in Canada, and, so, 
to plan future transportation services and facilities. Thus, it is important that you report 
as fully as possible on all the relevant trips taken by your household. 


The Statistics Canada interviewer has left this questionnaire for all members of your 
household to complete and will call back to pick it up on the date shown below. 


We would like you and your household to take a few minutes to answer the 
questions on the next few pages. Answers to most questions in this survey require a 
check in_the appropriate circle or the entering of a number in boxes 
provided [Ol1] . Atew questions ask you to write a brief answer in your own words. If you 
should have any difficulties with a question, note this in the space for comments and ask 
your Statistics Canada interviewer for assistance when he/she returns. | assure you that 


the information given by you is confidential and will be used for statistical purposes 
only. 


Thank you for your cooperation. 


Chief Statistician of Canada 


/ come 


Peter G. Kirkham 


Tole gkel Stas La etd eens eee 


PSU 
ant No 
Form No Docket No Survey Date Assign 


PICK UP DATE/TIME: 


Interviewer signature 


KED BY THE INTERVIEWER) Be 
a aaYONE YOUR HOUSEHOLD TAKE A TRIP OF 50 MILES OR MORE AWAY FROM HOME WHICH ENDED DURING 


IG 
PERIOD APRIL 1, 1977 TO JUNE 30, 1977 INCLUSIVE? (EXCLUDE TRIPS FOR THE PURPOSE OF COMMUTIN' 
TO WORK). 


Yes 1@ LEAVE QUESTIONNAIRE FOR 
be 2 O sag ‘ HOUSEHOLD TO COMPLETE 


INSTRUCTIONS 


t t t 
There is room below for you to record the detailed information equested for 8 difteren 
y 


to June 30, 1977 inclusive. 


include only trips which began In Canada, and were taken to places herent gets 
away from your home, except trips for the purpose of commuting oS w ie bape 
trip of that distance even If it does not involve an overnight stay. Also, include 

that distance to places such as a cottage, farm, chalet, etc.. 


art with the first trip which ended during the eae ei hn 1977 
the second trip during that period; then the third trip; “4 


We suggest that you st 
to June 30, 1977; then 


the housenold to make sure that all the trips that they took 


Check with every member of 
are recorded. 


ther, count Ht as one trip. 


if two or more persons go toge 


is survey comdected by Statistics Consds 
Te iy cnecty of tee Statistics Act. Chapter IS Stateter of Conads 


9767102 


Example 7.6 


TRIP 01 


WHERE DID YOU LIVE WHEN YOU TOOK THIS TRIP? Same city or town as now 
* IF NOT THE SAME CITY OR TOWN AS NOW, AND YOU LIVED IN CANADA, ENTER THE OR 
CITY OR TOWN NEAREST TO WHERE YOU DID LIVE, AND THE PROVINCE. 


nearest city or town 
province 


WHERE DID YOU GO ON THIS TRIP, THAT IS, YOUR DESTINATION? 

* IF YOU WENT TO MORE THAN ONE PLACE, ENTER THE PLACE THAT WAS FURTHEST country 

FROM YOUR HOME. 

* FOR ALL DESTINATIONS, ENTER THE COUNTRY. FOR CANADA AND THE U.S.A, ENTER eet 
THE PROVINCE OR STATE, AND THE CITY OR TOWN NEAREST TO YOUR DESTINATION. province stale 


nearest city or town 


WHICH OF THE FOLLOWING HOUSEHOLD MEMBERS, INCLUDING YOURSELF, TOOK THIS CHECK (,/) THOSE WHO WENT 
TRIP? | 


T1@) 
us© 
ns©) 
nO 
uO 
Te) 
in©) 
©) . 


HOW MANY OTHER HOUSEHOLD MEMBERS MOT LISTED ABOVE ALSO TOOK THIS TRIP? 
Enter number 


WAS THIS A WEEKEND TRIP? 


WAS THIS A VACATION TRIP? 


WHAT WAS/WERE THE REASON(S) FOR TAKING THIS TRIP? 


Other pleasure 
Business a] (a reasons 


CHECK ALL THAT APPLY > 


Visiting friends Personal 
or relatives... 12 O reasons 


Shopping in © 


9 HOW MANY NIGHTS WERE YOU AWAY FROM HOME OW THIS TRIP? 
None iw O) OR Nights 


| 8) WHAT WAS THE MEANS OF TRANSPORTATION USED TO TRAVEL THE GREATEST 
DISTANCE ON THIS TRIP? © 
Boat _ 138 


INCLUDE MOTOR HOMES, TRUCKS, AND VANS WITH AUTOMOBILE. 
CHECK ONLY ONE > 
Other in) 


WHY DID YOU USE THE MEANS OF TRANSPORTATION GIVEN ABOVE (QUESTION 10 
RATHER THAN SOME OTHER MEANS TO TRAVEL THE GREATEST DISTANCE ON THe 


TRIP? WRITE 4 BRIEF ANSWER > 


IN WHAT MONTH DID THIS TRIP END? CHECK ONLY ONE > 
April wi) 


HOW MANY OTHER TRIPS IDENTICAL TO THIS ON 
i gale E ENDED DURING THE PERIOD APRIL 1 


Enter number 


* COUNT AS IDENTICAL, ANY TRIP FOR WHICH ALL THE INFORMATION ABOVE IN QUESTION 2 TO 13 fS EXACTLY THE SAME. 
*00 NOT COMPLETE ANOTHER TRIP REPORT COLUMN FOR THESE IDENTICAL TRIPS. CONTINUE TO LIST ONLY TRIPS WHICH ARE DIFFERE 


145-77 


+ Lae 


Se 
TAIP 02 [STRIP 03_——~| STRIP 


Same city or town as now 'S 
a 145 Same city ae as now mC) Same city or town as now mC) 
clas For office on For office 
-—_—_ooooo ese onl 
nearest cily or town nearest city or town / nearest city or town pare. 
«TT ] Le 
province < 
province province 
country For office country 
country For office 
ese only se only 
_ ee ee eee 
province/ state province/ state province/ stale 
nearest City or town nearest city or town nearest city or town 
CHECK (//) THOSE WHO WENT CHECK (,)/) THOSE WHO WENT CHECK (\/) THOSE WHO WENT 


: | | 
i” O 182 O at O 


ws C) a) OO 
0 C) a) “ie 
iC) =) te 
r©@) nO Ae 
is C) aw ne) 
iC) is) ae 
r©@) is (_) © 


Enter number Ele [nll Enter number Enter number lh dull 


es 725 6) No m() 
es m1 Gy No mC) 


Yes 157 ©) No 
Yes : 138 Gy No 


~< 


~< 


Other pleasure Other pleasure Other pleasure 
Business 16) O reasons wi) Business 195 (S reasons Business ™ oS reasons m() 
Visiting friends Personal Visiting friends Personal Visiting friends Personal 
or relatives 162 reasons is) or relatives 196 O reasons or relatives 730 reasons m() 


Shopping 168 @® Shopping 197 @ Shopping 21 O 
None 166 a) OR Nights aia od None 268 we, OR Nights None. ™ &. OR Nights may 


mobile 


Bus im) Air mC) Other inC) Bus m() Air Bus mC) Air m() Other mC) 


For office 


Auto- 
Auto- Auto- 
im) Rail mC) Boat nC) mobile 202 Rail mobile 736 Rail m(_) Boat m(_) 


For office 


April ws) May nC) June nC) April mC) May nC) June nC) April mw) May mum) June rs) 
el Enter number ia al Enter number ude spon 


* PLEASE TURN THE PAGE TO RECORD MORE TRIPS. 


Enter number 


63400:77 . 18-0577 


oe = 


Examples 7./ to FPR, 


The questionnaire for the 1976 Survey of Fitness, Physical Recreation 
and Sport was completed directly by respondents. 


iz 


LO. 


The sponsor is clearly identified. 
The title is prominent. 


A letter from the Chief Statistician describes the purpose of the 
survey, asks for cooperation and assures the respondent of con- 
fidentiality of responses given. 


The control information is needed for administrative purposes 
and "cleaning" of computer tapes. 


Availability of a questionnaire in the other official language 
is stated, along with instructions for obtaining one. 


The first question is prominently marked. 


In order to eliminate excessive repetition of the questions for 
each category of answer requested, a somewhat novel arrangement 
of the questions and answer spaces was used on the interior pages. 


Another approach would have been to use a conventional tabular 

arrangement with questions down the left side and each category 
listed across the top of the table. However, this ruled layout 
would have been quite unappealing to respondents. 


There is little chance of respondents forgetting a question 
because of the unusually large numbers used to identify the 
questions. These large numbers are particularly useful for 
maintaining the correct sequence of questions 42 to 54, where 
some questions are in a vertical arrangement while others are 
side-by-side. The particular arrangement used depends on the 
amount and shape of the space required for each question. 


Although this layout appears extremely difficult to plan, this 
does not hold true. A draft of each question is prepared and 

cut out. The questions (each on its own slip of paper) are 
"floated" about on a page-size sheet of paper until the designer 
is satisfied with the result. This technique may also result 

in the sequence being changed to obtain a better fit on the page, 
so long as the logical sequence of questions is not upset. 


Each page (or set of two pages) has. a prominent identifier of 
the subject matter, together with a few statements which are 
intended to establish the desired frame of reference for that 
page (or pages). 
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Example 7.7 


Be Statistics Canada Statistique Canada 


>. . . ea 
Special Surveys Co-ordination Division 


1976 SURVEY OF FITNESS, PHYSICAL RECREATION & 
SPORT 


October 1976 


Dear Respondent, 


The purpose of this survey is to provide comprehensive data on Canadian participation in fitness, 
physical recreation and sport activities. It is being conducted for the Fitness and Amateur Sport 
Branch of Health and Welfare Canada. Data from the survey will be analysed by the Fitness and 
Amateur Sport Branch to aid in the planning and development of new programs, and in the 
investigation of exercise, physical recreation and sport participation of Canadians. 


The Statistics Canada interviewer has left this questionnaire for you to complete and will call back 
to pick it up on the date shown below. 


We would like you to take a few minutes to answer the questions on the next few pages. Answers 
to most questions in this survey require only a check mark in the appropriate box [Vj or circle (V). 
If you should have any difficulties with a question, note this in the space on the back of the form 
and ask your Statistics Canada interviewer for assistance when he/she returns. I assure you that 
the information given by you is confidential and will be used for statistical purposes only. 


Thank you for your co-operation. 


Peter G. Kirkham. 


\— 


Chief Statistician of Canada. 


This survey is conducted by Statistics Canada on behalf of the Fitness and Amateur Sport Branch, Health and Welfare 
Canada, under authority of the Statistics Act, Chapter 15, Statutes of Canada 1970-71-72. 


Age 


me is blade ra 
name 
emo TT TTT ETT TL ETt ty 
This questionnaire is available in both official 


“a8 a ee ) z i ee languages. If you require a questionnaire in the 
stile enamel Pact ees ule ( ) ao other official language please call this number 
questionnaire ans autre langue, 


tdéphoner au numéro. 


This questionnaire will be picked up by your interviewer on 


INTERVIEWER 


START HERE 


Do you euffer from any iliness or handicap which prevented your participation in physical recreation or sport activities during 
you 


the entire last 12 months? 


1a 
Ves Cy No — Go to Question 2 


Go to Question 40 


ae 
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Example 7.8 


This section of the Questionnaire deals with your participation in exercise activities within the last month. Although sport 
and physica! recreation activities are examined in the next section of the Questionnaire, any sport or physical recreation activity 
which you do primarily for exercise may be included in the Other (Specify) category of Question 3. 


hiave you engaged in any EXERCISE ACTIVITIES WITHIN THE LAST MONTH? 
105 lot 
Yes Y © No —® Go to Question 12 : 


Please indicate whether or not you have engaged in any of 
the following exercise activities WITHIN THE LAST 
MONTH? 


ee 


HOW MANY TIMES WITHIN THE LAST MONTH did you engage in 


each activity marked “YES” in Question 3? 


How many minutes do you usually 
spend doing the activity 
ON EACH OCCASION? 


Check “Yes” or “No” for each activity 


For those activities Times in the Minutes each ume 
marked yes in the last month 15 16 31 More 
answer questions or to to than 
4 through 9 less 30 60 60 
Walking for 
E é (Bye ee py Doss paisa 3s 138 140 14 
xercise Ne Y Yes 6.7.8.9 
124125 
Jogging/Running 107 y 108 pe D0 4. 5. De Gi. toys is 
No Yes 6.7.8.9 
Calisthenics (pushups, 109 } lio My Pari 126127 ae fae fe] ae 
situps, etc.) No Yes 6. 7. 8.9 
ore Ml y ee bo 45 | 128129 ae 151 12 183 
or exercise No Yes 6.7.8.9 
, 13 14 130131 
— > Do4.5 i] 155 fe Bie 
pope No Y Yes 6. 7, 8.9 L] L) 
Weight Lifting / ——— 
Training us y N60 Do 45: Hess (Bi: 158 160 161 
No Yes 6. 7. 8.9 
aan) (aa 
ere su y m = Does GATS The ‘Bie fa inl 
or exercise re} es 658.9 
eee eae 
136137 
Ski 118 10 _y Do 4.5 166 167 168 163 
1 i a No y Yes 6.7.8.9 
374875 
Other (specify) A re fey" fey BE ag 


i 


L____ WOTE: if you engaged in eny activity primarily for exercise which is not listed above, please Inciude It in the 
Other (Specify) category of Question 3. 


Please indicate the MAIN reason(s) you engeged in these exercises. Check one or more reasons 


30 
[_]* For Goop HEALTH: []* ENJOYMENT: 
good for my heart; to keep in shape; to stay in I like doing it; for pleasure and recreation; 
good physical condition; I can breathe better; etc. for fun; for excitement: etc. 
xm 
r GOOD FOR ME IN GENERAL: E-|* FOR THE RELEASE OF TENSION: 
makes me feel better; good for me; I feel for relaxation; to get away from 11 all; 
like it’s good for me; etc to unwind; etc 
co 
‘i TO LOSE WEIGHT: ies DOCTOR TOLD ME TO: 
to keep slim; I lke to keep my shape; physical therapy, post coronary recovery; 
I'm a litle on the heavy side; to flatten prescribed exercise, etc 


my stomach, etc 


[]™ OTHER REASONS; 
to be with my family; no parucular reason; 
compulsory at work or school; etc. 


In the last month, when did you usually exercise? (specify ) 


x) EY us 
C] Weekdays a Weekends [] Both 
12) 3 
office use only easy 
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Example 7.8 
ke al aa 


EXERCISE 


WITH WHOM did you USUALLY engage in each 
activity marked “Yes” in Question 3? 
AT WHAT INTENSITY did you USUALLY 
exercise in each activity marked “Yes” 
in Question 37 


AT WHAT TIME(S) OF DAY did you USUALLY 
engage in each activity marked “Yes” in 


| Question 3? 


WHERE did you USUALLY engage in each activity | 
marked “Yes” in Question 37 


Eom Only Check only one for each activity 
es :. Much perspiring/ slight Check one or more Outside 
n ith an perspiring/ Above eaitecence Morn- After  Eve- Al Public Commercial At work using no Other 
individual Rapid normale from nomial ings moons nings home recreation facility or or at special 
or group breathing breathing state facility private club school facility 


Dea OF OF Ge 0° Obs Roe. Lie eeeear 
Cera Oe Os OF” “hen ee iee ee alee 
ae Oe i oe Oe Ones Deis bie eee 
Be ee Gs GR) ied eis er ieee ee tine 
Ce OF) 2? Os) OF) CGS Elena silenemieasier 
Cl ee Gl an A a I eae hae 
Bl Ore Ce les ee eee | ee ei 
Cp) Gee eee eo ee ee ee ks ee oa 
oO oP oe c= | a eerie Se ets 


Check one or more. 


y, Which of the following wouid increase your interest in exercising more or beginning to exercise in the future 7? 
i 310 


Better/closer/less expensive facilities Bu Organized fitness classes with exercise programs 
prescribed by a professional or expert 


a More information on the benefits of exercising Ni iss I am sausfied with my present level of exercise 
a4 us 
[J More leisure ume [] Other (specify) 


316 
[] An individual or group to exercise with 


Comparing yourself to others of your own age and sex, would you say that 
a | ue 
3 you are LJ More fit 
Tr) 
CJ Less fil 


5) 
[] About the same fitness 


nu? 
[ office use only wet 
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Example 7.9 


THER LEISURE 


The following section deals with your 


ras ACTIVITIES 


participation in 


Please indicate whether or not you have engaged in 
any of the following leisure activities WITHIN 
THE LAST MONTH. 


HOW MANY HOURS IN A TYPICAL WEEK 
DURING THE LAST MONTH, did you spend doing 
each activity marked “Yes” in Question 42? 


Check “Yes” or “Wo” for each activity. 


For those activities Number of hours spent in a typical weck 
marked yes Less than 3-7 8-14 15 - 29 30 or more 
answer Question 43 3 hours hours hours hours hours 
; 678 679 (al 698 699 (aa 700 (al 701 ret 702 
Watching TV No Yes —— 
680 68) 703 704 705 706 107 
Listening to the radio No Vee aan hea [ ] a 
Listening to records tapes or 
682 383 708 703 no 11 nz 
cassettes No Yes es 
684 685 13 m4 ns ne 7 
Reading books for leisure No Ves a (a ea [J 


Reading newspapers, magazines 


for leisure 


Socializing or 


or relatives 


Engaging in a craft or hobby 


(photography, 


Doing yard maintenance - home repairs 


Playing bingo 


games 


Dancing 


. Cards, chess,other 


68 Cy [ 18 
No Yes cs 


Y 
Y 
y 
Y 
Y 
nee y sites ee [] 13 
Y 
Y 
y 
y 


= 
8 


Le ale 
Bape: 
Beer 
rey ig 


(a 740 a} 74) 
Gy Sle 


visiung with fnends 


> 
> 


3 
a 
= 


y 


691 ey 728 
Yes 
693 733 
cea we El 
Yes 
695 738 
san pe ae 
Yes 
697 [al 143 
Yes 


sewing, woodwork.etc.) CR, 


692 
No 


ve 


694 
No 


ON 


3 
= 
8 


= 
z 


alee Maleep ie! ejay 
Sheen c) a) ee) aa 


Y 


Please indicate whether or not you have attended 
or participated in any of the following leisure 
activities WITHIN THE LAST MONTH. 


||4 


Check “Yes” or “No” for each activity. 
For those activities Number of tumes within the last month 


marked yes | 2 3 4 5 or more 
answer Question 45 


HOW MANY TIMES WITHIN THE LAST MONTH 
did you attend each activity marked “Yes” 
in Question 44? 


Atiended a movie or other film oe Y nh > irs} real (7a pas fas ‘a ie, Be 
es 


Attended a musical performance or 


recital 


spectator 


circus, ZOO, or 


730 75) 769 770 ™m m 3 
a Or. No Dal Hag ab cs OR 
Attended a live theatre production, 
782 
a ballet or other dance performance Nie y ee = [ale Cale [a] “es L] ui fia] He 
Attended a sports event as a 
754 755 79 7e0 C] 781 nie} 72 [J "3 
Penh EO af 
Visited a cultural centre (museum. 
136 757 ras 7% Med 
an gallery, library, y es a re tel 
gallery. library.etc.) No Veo boa 
Visited a craft fair, festival, 
738 758 730 mI m2 m3 
hib: 789 
che Oe on Dae 


Participated in community acuvities 


such as church groups, service 


groups, etc 


No Y O%e > Dy OE ie Bas ae als 


Attended a class 


» One | rte Gir Le 
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Example 7.9 


This section collects information 
which is useful in grouping 


your replies with other Canadians 
In this survey. 


46 


Rix 
id Nova Scotia 
[a] me 
fe] 809 
ig Quebec 


47 


Were you born in Canada? 


ae Yes 


805 
No 


in which eat or territory 
of Canada were you born? 


fel 81 
Ee 
fae 
aq ai 
ie 815 
C] 816 


Newfoundland 


New Brunswick 


Prince Edward Island 


What language do you most often speak 
at home? 


23 026 
ral = English id Italian 
7] 
rer French Be Other 
(specify) 
bel ne German 


of 


ABOUT YOURSELF 


In which country were you born? 


ti? 
is] Great Bntain 
LI" 


United States 


Ontario 
Gy 
Germany 
Manitoba 
20 
[] Italy 
Saskatchewan 
2 
[J France 
Alberta 


aac 


Other/specify) 
Bnush Columbia 


NWT, Yukon Print country of birth 


When you were 13 years old what was the population 
of the city, town, village, municipality in which 
you were living? 


LM) 
ie 100.000 or more 


829 
[__] 30.000 to 99.999 


Lc) 
[]~ 1.000 to 14.999 


Pale 
eis 


less than 1.000 


don't know 


830 
[_]"15.000 to 29.999 


How tall are ee 


Eitri? Redeody) 


Feet Inches Cenumetres 


To what extent has your weight changed 
or fluctuated over the past 12 months? 


Check only one. 


ape 


Remained steady to 
within S pounds/ 
2.3 kilograms 


8% 
[] Fluctuated 6 to 
10 pounds/ 2.7 to 
4.5 kilograms 


[a\* 


Fluctuated more 
than 10 pounds/ 
4.5 kilograms 


in the last 12 months, has your doctor to 


53: 


%) 
Pia None 
[] be Less than 

$4,000 


[_]™ s4.000-6.999 


[_]™ s7.000-9,999 


41402-13-1 


Have not seen a doctor No 


What was your total personal income 
before taxes and deductions in the last 
12 months from all sources? (include 
earnings from employment, pensions, family 
allowances, interest, dividends, rents) 

(Please check one box) 


[_]™ 10.000 - 14,999 
C] ms 
[_]®’ 520.000-29,999 


[] zs $30,000 or over 


$15,000-19,999 


13-08-76 


How much do you weigh? 


check only one. 


—— = 
90 and less 91-100 101-110 111-120 121-130 
Ibs Ibs Ibs Ibs Ibs 
Mo A) 92 ui mae 
i) L LJ CL] L] 
ke kg ke ke ke 
40.9 and less} 414-455 | 459-500 50.4-54.5 550-59] 
131-140 141-150 151-160 161-170 171-180 
Ibs Ibs Ibs Ibs Ibs 
mS fry ur us 49 
L LJ LJ La L] 
kg k kg ke kg 
59.5-63.6 64.1-68.2 68.6-72.7 73.2-77.3 TETRIS 
— 4 
181-190 191-200 201-210 211-220 Over 220 
Ibs Ibs. Ibs Ibs Ibs 
iWrtetanep ids Ais Wer 
kg k ke kp ke 
82.3-86.6 86.8-90.9 i) 91.4-95.4 95.9-100.0 | Over 100 
be ak. = 


id you that you suffer from a heart related aliment? 


ak Yes 


How many automobiles are avaliable for use by 
members of your household? 


a) 
a [ive 
office use only 


THANK YOU FOR PARTICIPATING IN THIS SURVEY! 


67% 
One 


ar 
[] Three or more 
J 


None 


We would appreciate any comments which you wish to make on government 


programs reiated to fitness and amateur sport. Please ues the next page to 


do so 


nn 


= Goes 


Examples PRTG Se eer y: 


The Automobile Fuel Consumption Survey was conducted by means of the 
following steps: 


(a) Obtain a listing of licenced automobiles from the respective 
provincial governments. 


(b) Select a sample of these automobiles. 


(c) Telephone the owner of each automobile in the sample; complete 
a screening questionnaire for that automobile to verify its 
identity and whereabouts; and solicit the owner's cooperation 
with the survey. 


(d) Send the owner a diary-type questionnaire for the recording 
of all fuel purchases for that automobile within a specified 
four-week period. 


(e) Contact the owner by telephone about one week into the survey 
period to sustain motivation to complete the diary, and to 
answer any questions which the owner might have. 


(f) The diary was to be returned by mail (postage prepaid) at the 
end of the survey period. 


Page 89 shows the original screening questionnaire prior to the 
pretest. Page 90 shows the revised screening questionnaire which was 
actually used in the main survey. 


The differences between these two versions are interesting. Some 
problems which were originally anticipated did not emerge as problems, 
while others which were not anticipated did emerge, such as the 
number of times that ownership has been transferred between the time 
of registration and the time of the survey. 


Pages 91, 92 and 93 show the front few pages of the questionnaire 
which was in loose-leaf form in a 3-ring binder. 


Example 7.10 


Bb Statistics Canada Statistique Canada 


Identification No 


Gammon 


89 - 


[conF DENTIAL when contaall 
} 


AUTOMOBILE FUEL CONSUMPTION SURVEY 
SCREENING QUESTIONNAIRE 


INTRODUCTION: er 1 WOULD LIKE TO MAKE SURE THAT 
INFORMATION ABOUT YOUR VE 
IS CORRECT. ae 


1 DO YOU PRESENTLY OWN A .. . (Read description below)? 


Model Year 


ae lt 


Model 


£26 S2RRL Ree eee 


Licence Plate Number 


ae Se ele 


Yes (The information is correct) 


Yes (One or two corrections required) 


No (Does not own this vehicle) 


2 INTERVIEWER: Make necessary corrections, THEN GO TO 8 


Colour Mode] Year 


3 DID YOU EVER OWN THIS VEHICLE? 
1 
Yes @ 


4 WHAT HAPPENED TO THIS VEHICLE WHEN YOU STOPPED 
OWNING IT? 


Has been sold/given away 
Has been scrapped .... 


Other (specify) 
5 WHEN WAS THIS VEHICLE SCRAPPED/ (Read “other” from 


question 4) ? 


mee oe Vee 


Month Year 


6 WHEN WAS THIS VEHICLE SOLD/GIVEN AWAY? 


Ce ae eae 


Month Year 
7 ARE THE NAME, ADDRESS, AND TELEPHONE NUMBER OF 
THE NEW OWNER AVAILABLE? 
2 
No O END 


END 


1 
Yes ‘eS Record below and END 


Name: 


Address: 


Telephone: 
8 IS THIS VEHICLE A VAN, TRUCK, TRAILER, CAMPER, MOTOR 
HOME, OR MOTORCYCLE? 


9 IS THIS VEHICLE USED: 


10 IN WHAT MONTH AND YEAR DID YOU OBTAIN THIS 
AUTOMOBILE? 
1 
July - Sep CG) 


1 
Jan - Mar €) 

£@. 1 
Apr-Jun Oct - Dec 


11 WHEN YOU PURCHASED THIS AUTOMOBILE, 


WASIT... 
‘O © 
A demonstrator? .. . automobile 
GO TO 13 


2 4 
A used automobile?. . ke) Don’t know ke) 


12 HOW MANY MILES (KILOMETRES) WERE RECORDED ON 
THIS AUTOMOBILE’S ODOMETER WHEN IT WAS PURCHASED? 


Kilometres 


See. 


A brand new 3 


Miles 
gah As, ts 
OR 
.©, 
Don't know... 


13 IS THIS AUTOMOBILE BEING OPERATED AT THE PRESENT 


TIME? 


Ke, he, 
Yes No END 


14 ARE YOU THE PRINCIPAL DRIVER OF THIS AUTOMOBILE? 


20, 
Yes e GO TO 16 No GOTO 15 


15 INTERVIEWER: Read explanation of part 2, and ask 
WHAT IS THE NAME, ADDRESS, AND TELEPHONE NUMBER 
OF THE PRINCIPAL DRIVER OF THIS AUTOMO- 
BILE? 


Name 


Address 


Telephone 


16 INTERVIEWER: Read explanation of part 2, and ask 
IS THIS YOUR CORRECT ADDRESS? 


Record changes 
below and END 


8, 
No 


ee . : 
Authonity — Statistics Act, Chapter 15, Statutes of Canada 1970-71-72 
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Example 7.11 


Statistique Canada 


PASSENGER CAR FUEL CONSUMPTION SURVEY 
when completed 


SCREENING QUESTIONNAIRE Authority — Statistics Act, Chapter 15, 


Statutes of Canada 1970-71-72. 
Identification Number 


[eae eh ae 


VEHICLE DESCRIPTION: REGISTERED OWNER: 
Make 


SSeS ee eee as 


Model 


Licence Plate Number 


Saag 


Serial Number 


INTERVIEWER: 


nee Total contact attempts made by phone 


OWNER # 1: 


1.DO YOU PRESENTLY OWN A ... (READ VEHICLE DESCRIPTION)? 3. ARE THE NAME, ADDRESS AND TELEPHONE NUMBER OF 
THE NEW OWNER AVAILABLE? 


i 3 
ge No (never owned) o END . © © 
Yes record below No END 


2 4 
Ore Ore 
Yes (corrections) 4 No (once owned) 2 


2 WHAT HAPPENED TO THIS CAR WHEN YOU STOPPED OWNING 
IT? 


Name . 


Sold/given away /traded in 


demolished in accident SG Telephone 


Scrapped << ie Notes: 
no longer roadworthy 


OWNER # 2: 


1.DO YOU PRESENTLY OWN A... (READ VEHICLE DESCRIPTION)? . ARE THE NAME, ADDRESS AND TELEPHONE NUMBER OF 
THE NEW OWNER AVAILABLE? 


3 
ee One Y 1G hnaenn N ‘ae 
L--.3 Ww 10 


‘Orr ‘Orr 
Yes (corrections) 4 No (once owned) 7) 


Name 


gel | HAPPENED TO THIS CAR WHEN YOU STOPPED OWNING 


Sold/given away /traded in ope OWNER # 3 


6 
demolished in accident &) 


as ‘ce 
no longer roadworthy 


OWNER # 3: 

1. DO YOU PRESENTLY OWN A... (READ VEHICLE DESCRIPTION)? . ARE THE NAME, ADDRESS AND TELEPHONE NUMBER OF 
3 THE NEW OWNER AVAILABLE? 

N @ END y 

lo (never owned) Sy 

Yes record below 


4 
go to 
No (once owned) O 2 


peat HAPPENED TO THIS CAR WHEN YOU STOPPED OWNING 


Name 


5 
Sold/given away/tradedin ....... Ores QUESTIONNAIRE 


6 
demolished in accident O 


Bhd ‘ile 
no longer roadworthy 


81. .= 


Lo~-o00re 
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[pow 


ee else alee |e ae ae | 


NeW 
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Example 7.18 


CONFIDENTIAL 


SURVEY OF NON HUNTERS OF MIGRATORY GAME BIRDS IN 


NEW BRUNSWICK DURING 1977-78 SEASON 
HELP US 


CANADIAN WILDLIFE SERVICE SER MEL EE UET 


We need to get a better understanding of the people who may have purchased a Canada Migratory Game Bird Hunting 
Permit in New Brunswick but did not hunt Migratory Game Birds. Your answers to the following questions will help us 
do this. All replies are strictly confidential. 


Instructions: Please answer the short questionnaire by placing a simple check mark (/) in the small boxes. 


1. (a) Did you get a Canada Migratory Game Bird Hunting Permit in 1977? (check one)... 


(b) Did you get a Canada Migratory Game Bird Hunting Permit in 1976? 


7 


(specify) 
(b) In 4(a) above 


fe underline the one most important reason for not buying a Permit or not hunting Migratory Game Birds in 
1977-78. 


Thank you very much for your cooperation. Please return the questionnaire today using the special postage paid envelope. 
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Examples 7.19 and 7.20 

The Client Satisfaction Survey was conducted entirely by mail. The 

respondents were a sample of persons receiving veterans' benefits, 

or their surviving dependants. 

Page 96 shows a page of the draft questionnaire which was pretested 

among some 50 veterans. It was prepared using only a typewriter and 

a felt pen. 

Page 9/7 shows the same page as it was revised for the main survey. 

1. Arrows clearly indicate the skip patterns. This was particularly 
important because many of the respondents are elderly and would 
likely have less than perfect eyesight. 

2. All entry spaces are clearly marked. 

3. The questions about politeness were re-worded. 


4. Questions are not crowded on a page. 


5. Each questionnaire was serially numbered for administrative 
purposes during collection and data capture. 


6. Key-punch codes are printed down the right margin. 


Example 7.19 


During the past year, have you visited an office of the Department 
of Veterans Affairs? 


If you answered GY ves, please complete this question; 
otherwise, please go directly to question 2. 


If you have visited: 


a) Was it difficult to find the office? 
b) Were the office employees polite? 
c) Were you kept waiting for a long time? 


During the past year, has someone from the Department of Veterans 
Affairs visited you? 


C) Yes C) No C) Not Sure 


If you answered Yes, please complete this question; 


otherwise, please go directly to question 3. 
If someone has visited you: 
a) Was this person polite? Opjret Sure 
b) If you were asked for information, 
did the person explain clearly why 
Ore Sure 


it was needed? 


If you were given information, 
did you understand clearly what 
you were told? Ort Sure 


Did you get an answer to all 
you questions? Orit Sure 


Example 7.20 


2345 


During the past year, have you visited a representative of DVA? 
2 


Yes No Not Sure 


L_____| 


Please go directly to question 2. 


Let’s think about your last visit. 


Was it difficult to find this person? 


Do you think this person kept you waiting too long? 


Do you think this person was kind? 


at a hospital 


Where was your visit? 1 


at a Legion Hall 
Did you get medical treatment? 


at a DVA office 
2 Yes O No 
iW 


elsewhere 


10 


2. During the past year, has someone from DVA visited you? 
1 2 3 


Yes No Not Sure 


eS 


Please go directly to question 3. 


Let's think about the last time you were visited. 


a) Did this person explain why he was asking you for 


information? 


Did this person give you a Clear answer to all your 
questions? 


Do you think this person was kind? 


Do you think you are visited often enough? 


Would you prefer that all your visits from DVA were by 
the same person? 


=> Oe 
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OTHER FACTORS AFFECTING QUESTIONNATRE DESIGN 


There are a number of other principles which apply to questionnaire design 
which Should also be considered. The degree to which they can be applied 
will vary from survey to survey, and there may be some cases where the ap- 
plication of some of these principles may conflict. For example, ease of 
completion and ease of data Capture may not both be attainable in a parti- 


cular design. The designer is then faced with the problem of developing 
the best compromise in the design. 


8.1 Survey Population 


In some cases, the members of the Survey population will have a particular 
characteristic which should be accommodated in the design of the 
questionnaire. 


The desired population may be located anywhere within a much larger popu- 
lation and it may be necessary to contact large numbers of the general 
population in order to locate (i.e., to screen out) those persons or cases 
which constitute the actual intended survey population. Examples of this 
type might be handicapped persons living with their families, manufactur- 
ing plants which use a particular process, etc. Provision for this 
Screening process should usually be incorporated into the questionnaire so 
that interviewers carry out the screening in the prescribed manner. 


In some type of surveys, a single survey may cover various areas of an 
organization's activity and having a questionnaire routed sequentially to 
each of the appropriate departments would unduly prolong data collection. 
To facilitate collection in mail surveys, it may be desirable to break 
down the survey into several discrete questionnaires and to send them to 
an appropriate senior official in the organization. This official can 
then send out each questionnaire to the appropriate department for com- 
pletion simultaneously, and can check the completed questionnaires for 
mutual consistency before returning them. 


Questionnaires intended for completion by elderly people should be 
printed with a moderately-large type size because many persons in this 
age group are likely to have somewhat impaired vision. 


8.2 Desired Level of Precision of Estimates 


It is necessary to know, in advance, the desired level of precision of 
tes required for analysis, so that the data collected by the 
vel of detail appropriate to the given sample 
The level of detail will vary with the charac- 


the estima 
questionnaire are at a le 
size and sample design. 
teristic being measured. 


In the study of certain activities, there may be several distinct sub- 


populations involved which are suspected (or known ) to participate in 
the activity to quite different degrees. Typical of this Situation 
could be television watching habits of different age groups within a 


Sete! 00 fat 


family or the travel patterns around an airport of airport employees vs. 
airline passengers. It may be desirable (or even necessary) to identify 
sub-groups which have similar characteristics. 


If the sub-population(s) can be identified in advance, the size of sample 
required to obtain a specified level of precision for the sub-popula- 
tion(s) will be smaller (and in some cases, significantly smaller) than 
if a sample were drawn from the entire population and the sub-popula- 
tion(s) identified subsequently by some screening process. 


8.3 Comparability 


The value of a survey is likely to be substantially increased if the es- 
timates obtained can be compared with other surveys of similar popula- 
tions which were conducted at different points in time (or in different 
locations), or with external administrative data. However, this is use- 
ful only when the definitions, subject-matter classes and units of measure 
are comparable. It is usually possible to provide for this comparability 
by using standardized definitions, units, etc., insofar as feasible when 
planning surveys. 


In the area of social and economic surveys, Statistics Canada has devel- 
oped a number of reference manuals containing standard classification sys- 
tems and definitions, as well as recommended wording for the more usual 
types of questions. These manuals are available to all researchers in 
Canada (see references at end of this section). 


Planners of other types of surveys, such as those dealing with technical 
data, should strive to use standardized terminology wherever feasible. 


8.4 Confidentiality and Sensitivity 


There are a number of areas where respondents may feel that questions 
intrude into their privacy and therefore be reluctant to or refuse to 
answer them. These areas include questions of age, education, income, 
business information, personal attitudes, anti-social activity, medical 
history, etc.  The*partrveularmreas ofssensitivity?! and! thes iatencity, oF 
refusal, will vary among different respondents and social groups. 


Where such data are necessary for the survey, there are several approaches 
that can be used: 


8.4.1 Location in Questionnaire 


Wherever possible, questionnaires should begin with reasonably simple, 
innocuous questions which are neither intrusive nor demanding in order 

to get the respondent involved in answering and to establish rapport. 
Sensitive questions may either be carefully located with related subject- 
matter areas, or grouped near the end of the questionnaire. When asking 
a person about his/her type of work, it will appear quite natural to in- 
quire about the range of wages or salary paid for this type of work and 


= OE oe 


where the respondent fits into this range. In cases where this technique 
1s not feasible, sensitive questions are delayed until the respondent has 
completed most of the less-contentious questions. If the respondent then 


refuses to cooperate further, a substantial amount of data will already 
have been obtained. 


Note: The sequencing of questions is dealt with more full in Sub- 
SeCLion 67205): 


8.4.2 Ranges vs. Specifics 


Respondents will usually have less objection to a sensitive question if 
the question is asked in a way which permits the respondent to identify 
the range within which the answer falls. For example, if a question on 
income asks the respondent to identify into which range his/her income 
falls (0-$4,999, $5,000 to $9,999, etc.), the response is likely to be 
better than where a specific income figure is requested. 


8.4.3 Importance of the Survey 


The degree to which respondents feel that a survey is important or that 
the results are likely to affect them, will have a bearing on their 
willingness to answer sensitive questions. Consequently, it is useful to 
identify clearly the purpose of the survey in the introduction to the 
questionnaire or have it described by the interviewer. 


8.4.4 ) Assurance of Confidentiality 


Many respondents are willing to provide data on condition that the re- 
search organization can provide assurance that the respondent's identity 
will not be revealed or that the data will not be released to other 
parties, such as business competitors. 


In general, the response rate increases when respondents feel assured that 
the survey organization can and will maintain the promised confidentiality 


Anonymity and confidentiality are sometimes misunderstood by respondents. 
Anonymity exists only where the identity of the respondent is not entered 
in any way on the questionnaire and is not available to anyone, including 


the research organization. 


8.5 Method of Data Collection 


Each of the principal methods of data collection has somewhat different 

needs which should be reflected in the design of the questionnaire used. 

Consequently, the method of data collection should be decided before the 
’ 


questionnaire is designed. 
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For mail and drop-off (i.e., self-completed) questionnaires, all necessary 
explanations, definitions and completion instructions should appear on the 
questionnaire itself or on an accompanying pamphlet. In most cases there 
is no opportunity for supplementing the printed instructions with spoken 
instructions. The wording used in these questionnaires must be easily 
understood and the design should anticipate all problems which are likely 
to be encountered by respondents. In summary, the respondent receives the 
package and thereafter is "on his own". 


For surveys which are conducted by an interviewer, the questionnaire need 
not contain many of the general instructions and defintions, as these can 
be provided on separate reference sheets or an interviewer's manual for 
reference as needed by the interviewer. These questionnaires need con- 
tain only essential completion instructions, such as skip patterns where 
certain questions do not apply to all respondents. Questions can be more 
complex and specific questionning techniques (e.g. probing) can also be 
Specriied on the questionnaire: 


For face-to-face interviews, the quéstionnaire can contain questions 
which are not actually asked but are completed by the interviewer, based 
on observation. Examples of this type of question include: type of house 
construction, evidence of maintenance, whether street is paved, etc. The 
questionnaire can also be designed for the use of flash cards or other 
exhibits by the interviewer. One example of the use of flash cards is, 
where respondents are given a card bearing a printed list from which they 
are asked to make a selection, usually where the list of items is quite 
long or the items are quite complex. 


Questionnaires for telephone interviews are generally similar to those 
used for face-to-face interviews, but do not contain questions based on 
personal observation. As the interviewer can arrange to have relevant 
definitions, explanations, etc., arranged around the telephone, the ques- 
tionnaire itself need contain very few instructions to the interviewer. 
For surveys containing only a few simple "Yes/No" questions, it may be 
feasible for the telephone interviewer to use a tally-sheet on which 
responses to questions are entered, instead of individual questionnaires. 
These tally-sheets would be similar to questionnaires, except that entry 
spaces would be arranged in a columnar layout, with a column for each re- 
spondent. However, tally-sheets can be used only where the range of an- 
swers can be determined in advance and the answers are quite short. 


8.6 Physical Convenience 


The environment surrounding the completion of a questionnaire is usually 
quite different from the office environment where it was planned. Con- 
sequently, the problems encountered during completion, either by the 
respondent or by an interviewer should be considered carefully by the 
designer. A good example of this is a diary-type questionnaire which 

has entries made on it frequently by respondents under a wide variety of 
environmental situations. Most diaries are in the form of a small book- 
let, in order to achieve the minimum handling problem for respondents, at 
the cost of greater data capture time required in the processing office. 


ae 


In general, the convenience of com 
greatest priority, 

Len ware Crit Ca ly to 
factors to control. 


pletion and handling should be given 
because the response rate and accuracy of data collec- 
the success of a Survey, yet are the most difficult 


Diary-type questionnaires have been used successfully in household expen- 


diture surveys, radio-listening and TV-watching surveys, automobile fuel 
consumption surveys, etc. 


8.7 Response Burden 


Survey sponsors seldom estimate the time required by each respondent to 
complete a questionnaire. When this amount of respondent time is multi- 
plied by the number of respondents involved, the total time involved can 
become substantial. In view of this response burden and its potentially 
negative effect on quality of response and rate of response, it makes 
good sense to consider those factors which make this burden excessive or 
even unnecessary. These factors include: 


8.7.1 The Number of Questions 


Is each question necessary? Could some of the data be obtained from ad- 
ministrative or other existing records, or from a previous survey or 
Surveys in the same general area? 


Gane, Glhe Leneth vor Complexity of Questions 


Are the requested breakdowns of data necessary? Will they actually be 
used in tabulation? Would a number of simpler questions be easier to 
answer and, at the same time, provide adequate data? 


8.7.3 The Difficulty of Completion 


Will the respondent (particularly a business respondent) be required to 
rework existing data or to search through old records to satisfy the ques- 


tionnaire? Or will the respondent be expected to recall details months or 


years aiter the event? 


8.8 Editing of Completed Questionnaires 


Editing is the checking of returned questionnaires for Cag TS ee 

reasonableness and internal consistency. It can be done, either 
aires, or by computer after the entries 
tape (or both manually and by computer). 


Logie; 
manually on the actual questionn 
have been punched onto cards or 


ems which must be mutually consistent should 


For manual editing, those 7° ble them to be cross-checked 


be so located on the questionnaire as to ena 
easily (i.e., quickly). 


atu 


Editing is more likely to be needed for mail questionnaires than for 
questionnaires completed byinterviewers, because in mail surveys, ques- 
tionnaires are more likely to be returned only partly completed or even 
completely blank. In the case of personal interviews, respondents may 
refuse to answer some of the questions. 


In some cases, a perusal of completed questionnaires may reveal that a 
respondent (or even a trained interviewer) has misinterpreted questions 
in an obvious manner or placed answers in the wrong entry spaces. It may 
be possible to.correct Lhese errors by aotecence from the answers 21veu 
to other questions, before the questionnaires 20 to, davascaptuce. 


8.9 Coding 


Codes are symbols (usually digits) which are used to identify categories, 
ranges, etc:, to facilitate tabulation. -In surveysswhere the) categories. 
ranges; GLC... are knownm,in- advance; 1t-1s- usually desirable fo havesthe 
appropriate code pre-printed beside each entry space on the questionnaire. 
This can eliminate the need for manual coding. However, the decision to 
pre-print or not to pre-print codes on the questionnaire depends on a num- 
ber of circumstances unique to a particular survey, such as the number of 
open-ended questions where the nature of the answers cannot be anticipated. 


The process of coding consists of manually converting the answers provided 
by respondents into prescribed codes. The entry spaces for these codes 
are generally arranged in a narrow column down the right edge of the 
questionnaires or on a separate coding sheet. 


Example of a coding column on a questionnaire: 


Province OH 


Occupation OO 


Example of a coding sheet, using the same questions as above: 


Provie -OcenD:: 
ZO 21: 22. DSA 
a) a) = er ea ie 


In surveys where the frequency distribution is not known in advance, the 
code categories may not be determined until a tally has been made of the 
responses on the first 5% or 10% of the completed questionnaires. This 
approach requires that these questions be coded manually. 


81D Data Capture 


Most surveys are designed to collect answers which can be tabulated. This 
requires that every item of data which is to be tabulated must be read 
from the questionnaire and transcribed onto a punched card, magnetic tape 
by an operator at a keyboard, or transcribed manually onto a code sheet or 
tally sheet. 
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Where data are keyed directly from questionnaires, or code sheets, the 
machine operators do not have time to read indivi 
Instead, they concentrate on designated entries 

instructions. : 


dual responses for sense. 
codes and related keying 
For this reason, data Capture entries should be located in 
a standardized location throughout a questionnaire, preferably separated 
by open space from other printing. (See sub-section 7.7 for more complete 
treatment of data capture considerations.) Operators usually prefer to 
have entry spaces arranged in columns, with the card column or field iden- 
tifiers suitably identified in small numbers near each entry. However, 
there are many possible variations, and the questionnaire designer should 
consult with the data capture staff before completing the design. 


For manual transcription of data onto code sheets or tally sheets, most of 


the same principles apply, as for keying, although it is possible to do 
editing at the same time. 


8.11 Tabulation and Analysis 


In order that questions be structured to collect data in the desired for- 
mats, ranges, etc., the intended types of tabulation and analysis should 
be known when the questionnaire is designed. It is often useful to cons- 
truct dummy tables with detailed column headings and line captions to 
enable the researcher to identify all the variables which will be needed 
to carry out the intended tabulations and cross-classifications. If this 
is not done in advance, the researcher runs a risk of learning too late 
that a critical type of variable was not collected and, therefore, that 
the intended tabulation and analysis cannot be carried out completely, or 
that a particular variable was collected but was not needed to produce 
the desired tabulations. Unnecessary questions impose needless response 
burden on respondents. 
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SOURCES OF ERROR IN SURVEYS 


As a Starting point, it is important to understand the two major classifi- 
cations of survey related errors. Survey related errors may be either 
random or systematic. Random errors have no decided pattern and do not 
favour a particular result. Due to this ‘randomness’ in the-ervor, “ail 
Sedge have an equal chance of occurring. Systematic error (also called 
bias") occurs where there is a tendency to collect, process or interpret 
data in a way that favours particular sets of results. The effect of this 
systematic error is often cumulative and usually results in significantly 
distorted findings which deviate from the true or real values. 


There are several potential sources for both types of error in a survey. 
The following describes the likely sources to examine when trying fo re- 
duce the systematic error in a survey. It will become apparent, from the 
nature of the sources, that the likelihood for error can only be minimized, 
and never eliminated. 


9.1 The Data Collection Method 


When deciding how and where the data are to be collected in a survey, it 
is essential to carefully consider the objectives and the information re- 
quested. Data collection methods are not usually interchangeable and the 
researcher must be certain that the most appropriate method and location 
have beenchosen. For example, a complicated attitudinal questionnaire 
may best be handled by a trained interviewer in face-to-face contact with 
the respondent in the respondent's home. This would make it easier for 
the respondent to understand extensive instructions and would provide him 
with an ‘'unthreatening' atmosphere. If research is required of the res- 
pondent (e.g. finding copies of fuel oil bills, or assembling details of 
a business operation) a surprise on-the-spot personal interview will not 
collect the precise type of data needed. A respondent will be inclined 
to offer guesses rather than keep the interviewer waiting while he rum- 


mages for the required receipts. 


Consider a detailed interview about political views of housewives to take 
place in a large shopping mall. A housewife surrounded by her children 
may not provide thoughtful detailed answers. This is a good example (and 
not necessarily an uncommon one) of an inappropriate location for such 


a study. 


The choice of an inappropriate collection method and/or location will be 
the sources of a great deal of systematic error or bias. 


There are a number of collection methods and combinations of methods from 


which to choose. A few samples are: 


- telephone interview 

- face-to-face interv1ew 

- interviewer drop-off and p . 
mail-out/mail-back self-enumeration . 

- combination interview and self-enumeration. 


ick-up with self-enumeration 


= Lees 


Unfortunately they all have shortcomings. Even the most appropriate 
method comes with built-in sources for several other types of error. 


The following is a list briefly describing some of the types of systematic 
errors that can occur in the interview method. 


Interviewers have been known to complete questionnaires 
themselves and submit them as completed interviews. 
(This is particularly common when they have encountered 
problems in locating the individual selected by the 
sampling procedure. ) 


Interviewers can fail to follow the specified sample 
plan by approaching the 'house next to the sample 
house' (i.e., the one that doesn't have the snarly 
watchdog) or settling for the 'wrong’ person to 
interview. 


Interviewers sometimes fail to follow instructions on 
how to ask questions (often due to their lack of under- 
standing of the instructions). They can, by their 
behaviour when asking the questions, by intonation or 
by commenting on the questions or answers affect the 
answers they receive. As well, they can omit questions 
by inadvertently misreading skip instructions. 


The interviewer-respondent interaction can bias re- 

sults. The interviewer, from his/her appearance or 

attitude, may unknowingly lead to a certain pattern 

of résponses or discourdge responses even to a point 
of complete non-response (e.g., an anglophone inter- 
viewing a francophone). 


Interviewers are sometimes forced to 'make decisions' 
or adopt improper procedures during an interview due 
to unclear instructions or instructions whiclr do not 
prepare the interviewer for all possible awkward 
Situations. 


Finally, interviewers can make recording errors by 
writing down incomplete or wrong answers even if 
the questions have been asked and answered correctly. 


Many of these sources of error can be minimized by careful control proce- 
dures, such as: careful drafting of interviewer training manuals; careful 
and thorough training of interviewers; close supervision during the first 
few interviews (by a supervisor accompanying the interviewer for immedia- 
te correction of errors); monitoring of all’) interviewers (e.g. by verifi- 
cation procedures such as re-interviewing a sample of each interviewer's 
respondents); and interviewer de-briefing at the end of data collection 
to detect apparent or real problems the interviewers met. 
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A self-enumeration type of questionnaire de 
written instructions and the motivation of 
follow them to obtain correct responses. 


pends on the clarity of its 
the respondents to read and 
If the instructions are not 
clear or are too complex or are long, respondents may not follow them. 
As mentioned elsewhere, it is frequently helpful to repeat instructions 


in a self enumeration questionnaire directly before the question to which 
the instructions apply. 


In a diary-type enumeration questionnaire, bias can arise from the effect 
of the questions on the behaviour which is being measured. Respondents 
asked to record purchases or other behaviour may feel that they should be 
conscientious. They may buy items different from usual or watch more or 
different TV, in order to have more to report and thus show themselves 

as co-operative respondents (particularly if an incentive has been offered). 
At an extreme, there might be outright falsification of the items reported. 


Rates of non-response are usually higher for self-enumeration question- 
mnaires, particularly where there is no personal contact by an interviewer, 
such as in mail out/mail back surveys. For many mail surveys, response 
is likely to be less than fifty percent from a single mailing of the 
questionnaire. The difficulty then arises in determining whether the 
characteristics under examination differ between those who do respond and 
those who do not respond. It may be possible to check demographic 
characteristics against Census data, and introduce weighting procedures 
to correct for deviations from general population characteristics. It 
may even be possible to do the same for behaviour by comparing results 
with data from other sources. It is probably not possible to correct 

for differences in attitudes, so that a bias of some kind and unknown 
extences likely to pexpresent . 


9.2 The Collection Instrument 


Assuming that the most appropriate method and location of collection have 
been established, and the sources of error from them minimized, the next 
source of errors to be examined is the collection instrument. (Note that 
the design of the collection instrument will be affected by the choice of 
collection method and, that the choice of method of collection —— be . 
affected by the design of the collection instrument. There is no best 
ordering of these two procedures. Each must be accomplished with consi- 


deration of the other.) 


Error,particularly systematic error, can occur in the collection teas 
ment (i.e. questionnaire) from wording of instructions pines nan 
section 5), sequencing of instructions and items (see section an rom 
the layout of the instructions and items (see section 7). 


As well, bias can come from unreliable or invalid sag gee -odlinere rts 
(see section 4). Frequently, this is an error source that eae 
not considered serious enough to deserve the RSS Bie = ate 
merit. Unless the survey designer 1s prepared re do eceer oe p 

and testing work essential for a new set of words or ph 
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attitudinal scales, he should make use of scales already developed and 
tested by others. If the researcher develops his own attitude-scale 
questions, he must test to ensure that the scales are reliable (i-.e., 
consistent among matched groups) and valid (i.e., measure what they were 
designed to measure). Without this testing, the major error of measuring 
an attitudinal item more than once, is likely to oceur; This can, lead to 
a false impression of the intensity of an attitude or to the conclusion 
that more than one aspect of the attitude has been measured. For example, 
in studying interpersonal communication, it could be misleading to accept 
measures of attitude towards cooperativeness, friendless, and lack of con- 
flict, as if they were separate items, when further analysis might demon- 
strate that respondents do not perceive much of a difference in the mean- 
ings of these terms. Sometimes, the likelihood of lack of differentiation 
by respondents may not be obvious, and a variety of statistical techniques 
(e.g., multidimensional scaling techniques, such as discriminant analysis, 
cluster analysis, factor analysis) are available to help reveal underlying 
groupings of items and the extent to which they are effectively measuring 
the same concept or dimension. (Reference: Wish, M. and Kaplan, S.J., 
"Toward an Implicit Theory of Interpersonal Communication", in Sociometry, 
19774 VOL. CU NOm: So 2DD.u oo 24) 


If data are to be collected from two or more language groups, and the 
questionnaires, interviewer training manuals, coding manuals, etc., are 
prepared initially in one language it is essential to consult frequently 
with translators on anticipated problems in translation. Subtle shades 
of difference in meaning can occur or there can be outright mis-transla- 
tions. For example, in one survey the term 'nursing' was translated to 
the French term 'allaitement' (which means breast-feeding babies). The 
survey was examining nursing in the sense of caring for the ill and the 
translation should have been a phrase such as ‘profession de 
garde-malade'. 


9.3 Selecting the Respondent 


The respondent unit and sample selection procedures must be carefully de- 
termined in order to avoid bias. 


The respondent unit must be suitable for the research objectives: for ex- 
ample, proxy response (e.g., from the housewife for all other family mem- 
bers) may not be suitable if the respondent does not know the experiences, 
behaviour or attitudes of those for whom the response is being given. 


Bias in survey results can sometimes be traceable to complete non-response 
from respondents, because they were not available at the time of the 
interview or were untraceable. For example, single males aged 18-25 are 
at home much less than married males aged 45 or over. If the survey bud- 
get cannot afford very many repeated trips to the homes of young single 
males to find them at home for interview, they may be underrepresented in 
the survey. Complete non response may occur in business surveys because 
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small businesses cannot affort the time or expense of compiling the in- 
formation requested, or they may keep their records in less detail than 
larger businesses so that they are unable to compile the information 
requested. Some of the procedures used to anticipate and correct for 
this bias include weighting in the sampling procedures or the use of 
"not at home' weights such as in the Politz technique. 


Another source of error in the selection of respondents can come from 
the sample frame used. If the frame is incomplete, part of the popula- 
tion will be underrepresented. For example, surveys which use telephone 
directories as the frame omit households without telephones, new list- 
ings, and unlisted households. 


Error can be introduced if interviewers or other field staff fail te 
follow the specified sampling plan. 


In business surveys, questionnaires are often sent to the 'President' or 
the 'Accountant' if no other company officer is known. There is no gua- 
rantee that these particular officers will be able to respond to all the 
types of questions that are asked of businesses. For example, few pre- 
sidents would be able to provide (without the aid of subordinates) de- 
tails on shipments etc. Also with such general addressing there is lit- 
tle control over the actual choice of respondent. 


The same problem occurs with self-enumerated questionnaires. Bias can 
be introduced by the lack of control over selection of respondents 
especially when general addressing such as 'to the householder' is used. 
Respondents may consult with others to refresh their memories or may 
test their responses on others and thereby be influenced/biased by the 
memories and opinions of others. As well, there is no way of ensuring 
that the individual who responds will, in fact, be the person to whom the 
questionnaire was addressed. 


Further bias arises from the fact that those most interested in the sub- 
ject of the survey, those with strong feelings for or against the topic 
under research, those with time available (e.g., retired people) and 
people who lead relatively inactive lives are more likely to reply to 
questionnaires than is the remainder of the sample population. In order 
to maximize response from the whole selected sample, reminder letters, 
second and third mailings of the questionnaire, telephone follow-up re- 
minders and possibly even telephone or personal interviews can be 


employed. 


9.4 Conclusion 


As indicated in the introduction to this section, the ae eee os sys- 
tematic errors can be minimized but never eliminated. LE : < - - 
apparent from the descriptions that to eliminate error 40 ri P path y 
the collection instrument, one would have to eerie pe eae ae 
itself, \To eliminate “interviewer, Crrer,, Be would have to a 


interview method and so on. 
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Another frustrating trait of these systematic errors is their elusiveness. 
Their effects on the accuracy of the results are virtually impossible to 
measure, primarily due to the inability of the researcher to control the 
activities that give rise to them. “Thus, researchers must settle for weak 
estimates of the systematic errors in’ their results. 


iGs 
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THE PRETEST 


Just because all principles described in the previous sections have 
been followed, there is no guarantee that the proposed questionnaire 
will fully satisfy the objectives of the study, no matter how 
conscientious the researcher has been in designing the questionnaire. 
There are almost always unforeseen problems that occur in the 
administration of a questionnaire. As a result, it is essential 
that a pretest of the questionnaire be implemented for all new 
Surveys and for already existing Surveys on which substantial 
modifications have been made in order to determine whether the 


research objectives are likely to be met by the proposed question- 
naire. 


Some aspects of the questionnaire that the researcher may test are 
the following: the wording, sequence and layout of the questionnaire 
to determine whether the questions and their flow are understood 

by respondents and interviewers; the necessity for inclusion of 
particular questions; the choice of types of questions; the use of 
specialized questioning techniques such as ranking or rating 
questions; the structure and definition of response categories; 

the degree of usage of the "other" category in questions; the 

ease of administration of the questionnaire; the time of adminis- 
tering various sections of the questionnaire; translation of the 
questionnaire; the possibility of bias in the questions; the nature 
of ethnic, regional or linguistic differences; the reasonableness 
of the questionnaire with respect to its demands on the respondent; 
the suitability of the questionnaire for measuring the concepts on 
which measurement is required; letters of introduction or intro- 
ductory procedures; and the suitability of the method of collection. 


A pretest should be done on a small sample of respondents (usually 
twenty to thirty) from the target population. It is preferable that 
the respondents be selected from the various subpopulations of the 
target population where differences or problems are likely to occur. 
Possible variables for definition of the test subpopulations are 
geographic region, educational background, age, sex, language, size 
of firm and type of industry. Depending on the particular purposes 
of the pretest, either a probability or a non-probability sampling 
scheme may be required for the selection of respondents, although 

in most cases, the latter is employed. One possibility is to use a 
focus group discussion of the questionnaire as a part of the pretest 


procedure. 


The method of collection used for the pretest should be identical to 
that planned for the main survey. However, a personal interview a ! 
recommended for at least a portion of the BEEP bee pie ge oa a 
the interviewer can then record the respondents’ reactions, bot 
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ees eee ee eee the interviewer can Sieeeee peepee j 
that the respondent had, the interpretation of questions an ee 

dso on. These difficulties can then be discussed wit 
RE the questionnaire, for example, in the context of a 


as well as their own suggestions and impressions. 
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meeting among the questionnaire designer and the pretest interviewers to 
debrief them on the interviews. For some pretests, it may be preferable 
to use experienced, skilled interviewers in order to maximize the use- 
fulness of the pretest. 


The 1974 Survey of the Newfoundland Resettlement Program, designed 
by the Statistical Services Division of the Department of Regional 
Economic Expansion included a pretest of its questionnaire and 
modifications based on the pretest results were made for the main 
survey. Example 10.1 below shows three questions from the pretest 
questionnaire and the revisions made to them for the main survey. 


Example 10.1 


Survey of the Newfoundland Resttlement Program (Department of 
Regional Economic Expansion) 


Pretest: 
26. HOW LONG WAS IT BEFORE YOU COULD MOVE INTO YOUR PERMANENT DWELLING? 


NUMBER OF WEEKS 
27. WHAT WAS THE REASON FOR THE DELAY? 


1. COULD NOT FIND A DWELLING 

2. DELAYS IN MOVING HOUSE 

3. LACK OF SERVICES FOR DWELLING 
4 


OTHER, SPECIFY 


WHERE DID YOU AND MEMBERS OF YOUR HOUSEHOLD STAY IN THE MEANTIME? 


Main Survey: 


HOW LONG WAS IT BEFORE YOU COULD MOVE INTO YOUR DWELLING? 
LENGTH OF TIME 
WHAT WAS THE REASON FOR THE DELAY? 


DELAYS IN MOVING HOUSE - SERVICED LAND NOT AVATLABLE 
COULD NOT FIND DWELLING pO HE 7 “SP BCGure Y 
LACK. OF SERVICES 

. BUILDING/RENOVATING - UNKNOWN 


WHERE DID YOU STAY IN THE MEANTIME? 


1. RENTED A DWELLING . STAYED IN HOTEL/MOTEL 
2. STAYED WITH FRIENDS/RELATIVES = (OTHER, vor mc le. 
3. BOARDED 


- UNKNOWN 
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The analysis of the pretest Suggested a more specific answer category 
(question 26), the inclusion of more response categories based on 
sufficiently frequent write-in responses in the "other" category 
(question 27), and the closure of an open-ended question based on 

the main write-in responses to the open-ended question. In addition, 
the pretest suggested the reordering or exclusion of some questions. 


The 1978 Annual Work Patterns Survey, sponsored by the Labour Force 
Survey Division, Statistics Canada, served as a pretest for the 1979 
survey. Example 10.2 below shows two Interviewer Check Items used 
as filter questions to determine those respondents eligible to 
answer the question, "What was the main reason ... did not look for 
work in 2?" These two items were subsequently dropped in 
the 1979 survey because of the frequency of interviewer errors in 
completing them and because of respondent difficulty in providing 
the information (especially by proxy response). 


Example 10.2 


Annual Work Patterns Survey (Labour Force Survey Division, Statis- 
ties Wanada ) 


24 INTERVIEWER CHECK ITEM: 


00 
ON AVES Sim ergs ase ee Ae ee '& go to 28 
® OTHERWISE: 
A. For each month marked in 13 mark the O 
corresponding month below and beep go toB 


WPOFOXO"OFO"O"O"0"0"'0"0 


B. For each month marked in 21 mark the @ 
corresponding month above and go to 25 


25 INTERVIEWER CHECK ITEM: 


“ 
@'/f all months marked in 24 go to 26 


@ OTHERWISE: 


For each month not marked in 24 ask: WHAT WAS THE sist 


.. . DID NOT LOOK FOR WORK IN —_________—-* 
os aN (repeat month(s) NOT marked in 24) 


J 


F M A M J J A S O N D 
Enter 01 02 ~—s«833 06 


ray es ral Cats ae oa 


code 


ee Gr 


In example 10.3, the definition of full-time work was included in 

the question itself in the 1979 survey while the 1978 survey provided 
the definition in the interviewer's manual. This was done to make 
the questionnaire more self-contained and to reduce potential -errer 
in the response to the question. 


Example 10.3 


Annual Work Patterns Survey (Labour Force Survey Division, Statistics 
Canada) 


1978 Survey: 


12 WAS THIS WORK IN 1977: ENTIRELY FULL-TIME, ENTIRELY PART- 
TIME, OR SOME FULL-TIME AND SOME PART-TIME? 


1 
Entirely full-time OEE 2s Pere san Ore ) go to 28 


2 
Entirely part-time .... ee ee a © go to 18 


3 
ee ee O go to 16 


Some full-time and some part-time 


1979 Survey: 


12 WAS. ...’“S WORK IN 1978 ENTIRELY FULL-TIME, 
ENTIRELY PART-TIME OR SOME OF EACH? BY FULL-TIME 
| MEAN 30 HOURS OR MORE A WEEK. 


1 
ENntirelyetulletiMCee> vee een Bee eee ‘S. 

2 
Entirely Dartctline@s. 7 onc, pon sakes one aT en a ee Ge) 


Some full-time ‘e) 
ANGSORTE: PanltiMe: Gee ce eee ee ee eee 


In the 1978 National Survey of Driving Habits, sponsored by the Road 
and Traffic Safety Branch of Transport Canada, the decision to change 
the layout of the diaries contained in three-ring binders from sideways 
to vertical to facilitate respondent handling and keypunching 

resulted from the pretest. 


Depending on the complexity of the overall survey methodology, and 

the time and budget available, it may be necessary to test more than 
the questionnaire component of the survey. Some procedures or 
concepts that may need testing are the following: the appropriateness 
of a survey to resolve the problem under consideration; the likelihood 
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of Controversy arising from the survey; the adequacy of sampling 
Instructions; the variation in the population (for purposes of 
sample size determination); the validity and reliability of 
attitudinal items; the efficacy of interviewer training; the 
effectiveness of the field organization; coding; appropriateness 
of the questionnaire for data Capture; and cost. Often, a small 
replica of the whole main Survey may be implemented to test its 


various components and their interrelationship. This is frequently 
called a pilot survey. 


It may be useful for the researcher (or research team) to be directly 
involved in the pretest/pilot survey to the point of accompanying 
interviewers or even conducting interviews. Much can be learned in 
the field concerning that which is realistic for implementation. 


After an evaluation of the pretest/pilot survey, revisions can be 

made to survey procedures and documents. In fact, the researcher 

may obtain evidence, as a result of the pretest/pilot survey providing 
him/her with a more realistic outlook on the potential of the main 
Survey. Objectives may even need revision. If substantial changes 
take place, another pretest/pilot survey should be carried out. 


Generally, respondents approached for the pretest/pilot survey 
should be excluded from the main survey. This may introduce a bias 
(which is usually minimal unless the size of the population is small) 
into the survey results, but is recommended as a public relations 
gesture to alleviate the potential burden of questioning respondents 
on the same subject matter on two occasions relatively close in time. 
Also, in the case of attitudinal surveys, respondents may become 
more educated in the concepts introduced in the pretest/pilot survey, 
a situation which may cause bias in the main survey. However, if no 
changes are made to the survey documents or procedures as a result 
of the pretest/pilot survey and depending on the sample selection 
procedures used in the pretest/pilot survey and the main survey and 
on the time reference period required in the main survey, it may be 
possible to include the results of a pretest/pilot survey in the 
results of the main survey. However, such inclusion is rarely done 


im actual practice. 


The pretest/pilot survey is an often-neglected quality control 
procedure. It will almost always suggest improvements or will at 
least give the researcher some assurance that the main survey, a 
much more expensive proposition, will likely proceed fairly 
efficiently. Of course, there 1s never any guarantee that all 
problems will be solved, but most major ones should be. A pretest/ 
pilot survey need not be expensive and need not require a great. 
deal of time for implementation and is recommended for all untried 


components of a particular survey. 
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